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The  objectives  of  the  study  are  to  Identify  and  document  the  actual 
impacts  observed  for  the  candidate  development  programs,  areas  of  poten- 
tial warranty  impact,  barriers  to  achieving  increased  beneficial  impacts, 
problem  areas  associated  with  warranty  application,  and  actions  that  OSD 
and  the  Services  should  consider  to  improve  the  application  and  imple- 
mentation of  equipment  warranties. 

The  analysis  consists  of  combining  the  results  of  many  Government 
and  contractor  management  interviews  with  contractor  responses  to  a 
standard  research  questionnaire  that  was  developed  to  provide  a common 
structure  for  recording  information  describing  contractor  experience  with 
warranties 

The  principal  findings  of  the  study  are  in  the  areas  of  (a)  warranty 
requirements  and  application,  (b)  impact  of  warranties  on  candidate 
development  programs,  (c)  impact  of  warranties  on  candidate  product 
designs,  and  (d)  conditions  leading  to  increased  warranty  impact.  It  was 
found  that  relatively  small  design  and  development  program  impacts  were 
occurring  as  the  result  of  current  warranty  application.  The  potential 
for  substantial  impacts  was  found,  however,  if  uncertainties  in  the 
warranty  acquisition  decision  process  were  eliminated  and  if  specifica- 
tion relief  were  granted.  Other  barriers  to  increased  beneficial  impacts 
were  found  to  be  the  constraints  imposed  by  limited  program  funding  and 
compressed  development  schedules. 

Based  on  the  study  findings,  conclusions  are  drawn  regarding  the 
warranty  impact  on  the  candidate  subsystems  examined  during  the  study. 

It  was  concluded  that  the  warranty  application  has  been  uncertain  and 
Imperfect,  that  warranty  options  are  presently  not  providing  strong 
design  and  development  incentives  for  achieving  improved  operational 
reliability,  and  that  while  warranty  actions  have  captured  contractor 
management  attention,  more  effort  could  be  expended  during  development 
to  achieve  the  warranty  objectives. 

Guidelines  for  future  policy  development  resulting  from  the  findings 
are  suggested  to  improve  the  application  of  warranties  to  electronics 
subsystems.  These  guidelines  relate  to  the  areas  of  warranty  integration 
into  design- to-cost  programs,  optimal  scheduling  for  warranty  decisions, 
warranty  requirements  definition,  specification  flexibility,  reliability 
data  and  predictive  models,  development  program  funding,  and  development 
program  schedule  flexibility. 
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FOREWORD 


During  the  past  few  years,  an  increasing  number  of  elec- 
tronics subsystems  have  been  developed  with  contractual  obliga- 
tions for  long-term  equipment  warranties  tied  to  the  achievement 
of  improved  operational  reliability  and  maintainability.  These 
new  contractual  techniques  for  guaranteeing  equipment  field 
performance  and  for  internalizing  some  of  the  attendant  risk  to 
the  equipment  contractors  have  been  a source  of  debate  between 
the  Government  and  contractor  Interest  groups.1  This  debate  has 
centered  around  issues  relating  to  contractor  financial  risks, 
reliability  prediction,  and  equipment  maintenance  roles. 

This  original  research  effort,  sponsored  by  the  Office  of 
the  Assistant  Director,  Defense  Research  and  Engineering  (Plan- 
ning), investigates  the  impact  of  these  early  warranty  options 
on  the  electronics  equipment  design  and  development  process. 
Because  it  is  in  the  engineering  development  program  that  most 
of  the  concerns  expressed  by  the  contractors  and  the  Government 
can  be  minimized,  the  results  of  this  research  provide  an 
important  input  to  further  warranty  policy  development. 

As  discussed  in  tne  Introduction,  this  study  is  limited  in 
scope  (eleven  electronics  subsystems)  and  depth  of  treatment; 
therefore,  some  of  the  findings  serve  only  to  identify  opportuni- 
ties for  more  extensive  investigation.  However,  the  findings  and 
conclusions  of  this  study  are  believed  to  be  transferable  to 
■future  Service  and  contractor  experiences  with  warranties.2 


Notably,  the  Council  of  Defense  and  Space  Industry  Associations  (CODSIA) 
and  the  headquarter  staffs  of  the  Services  and  the  Office  of  the  Secretary 
of  Defense  (OSD). 

2The  eleven  candidate  subsystems  comprise  the  total  (continued  on  next  page) 
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The  scope  of  the  research  is  also  limited  by  the  original 
task  order  to  an  investigation  of  the  impact  of  reliability  im- 
provement warranties  (RIWs)  on  equipment  design  and  development. 
However,  our  investigations  reveal  that  most  of  the  programs 
under  study  contain  additional  forms  of  potential  warranty 
obligations,  such  as  guaranteed  mean-time-between  failure 
(MTBF)  provisions  or  guaranteed  logistics  support  cost  (L SC) 
agreements.  Because  specific  effects  of  RIW  provisions  are 
often  impossible  to  isolate,  all  similar  requirements  are 
included  under  the  generic  term,  warranty,  for  the  purposes 
of  this  study.  Impacts  attributable  to  specific  types  of 
warranty  requirements  are  identified  whenever  a clear 
requirement-impact  relationship  exists. 

Perhaps  the  most  significant  observation  to  be  made  here 
is  that,  based  on  recent  research  into  allied  topics,1  the  re- 
search findings  are  unexpected.  These  findings  present  an 
interesting  insight  into  the  complex  forces  that  influence  the 
electronics  subsystems  development  and  acquisition  process. 


(contd)  population  of  electronics  subsystems  with  long-term  warranty  obli- 
gations that  were  available  for  our  investigation  in  October  1975- 
Several  additional  programs,  including  the  Army's  ARN-123  receiver  and 
ARN-124  distance  measuring  set,  the  Air  Force  C-l4l  Carousel  navigation 
system,  and  the  Air  Force  Ctnega  navigation  system,  were  negotiated  with 
warranty  contract  provisions  subsequent  to  the  establishment  of  the 
candidate  subsystems  for  this  investigation. 

1See,  for  example,  C.  David  Weimer,  The  Application  of  Design- to- Cost 
Acquisition  Policies  to  Selected  Electronics  Subsystem  Development 
Programs , IDA  Study  S-459  (1975). 
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EXECUTIVE  SUMMARY 


A.  INTRODUCTION 

This  study  report  presents  the  findings  and  conclusions 
of  a one-year  study  of  the  impact  of  reliability  guarantees 
and  warranties  as  measured  by  changes  in  electronics  subsystem 
design  and  development  program  effort.1  The  study  analyzes  the 
experiences  of  seventeen  contractors  who  participated  in  the 
development  of  eleven  electronics  subsystems  that  were  candi- 
dates for  reliability  guarantees  or  warranties.  These  elec- 
tronics subsystems,  shown  in  Figure  S-l,  comprised  the  total 
population  of  electronics  equipment  under  multi-year  warranty 
consideration  at  the  initiation  of  the  research.2 

The  research  effort  was  based  on  the  premise  that  contrac- 
tors with  potential  warranty  obligations  would  undertake  addi- 
tional development  efforts  that  would  influence  the  equipment 
design  and  change  the  development  program  effort  in  order  to 
improve  equipment  reliability  and  minimize  their  financial  risk 
under  the  warranty.  It  was  planned  that  the  study  would  identify 
and  document  those  initiatives,  and  that  insights  drawn  from  the 
findings  would  enable  new  development  programs  to  be  structured 
more  effectively.  Acquisition  policy  changes  or  initiatives 


1 Development  program  effort  has  been  described  in  terms  of  a percentage  in- 
crease or  decrease  over  current  expenditures  for  manhours  and  materials 
according  to  functional  elements  of  a standardized  development  program 
task  breakdown. 

20ne  possible  exception  is  the  Navy  APN-191*  absolute  altimeter,  which  was 
being  produced  under  a limited,  two-year  warranty  by  Honeywell,  Inc. 
Several  other  subsystem  development  programs  Incorporating  warranties 
were  planned  subsequent  to  the  selection  of  the  candidate  subsystems. 


No. 

Subsystem 

Status 

6/30/76 

Development 
Contractor^ ) 

1. 

ARN-118/119  TACANa 

Prod. 

Coll1nsc 

General  Dynamics 

2. 

ARC- 164  UHF  Radio3 ,b 

Prod. 

Magnavoxc 

Collins 

RCA 

3. 

APN-209  ABS.  Altimeter3 

Prod. 

Honeywell0 

Hoffman 

4. 

ASN-128  Doppler3 

Eng.  Dev. 

Singer-Kearfott 

Teledyne/Nyan 

5. 

SLQ-31/32(V)  EW  Suite3 

Eng.  Dev. 

Hughes 

Raytheon 

6. 

B-l  ECM  Suite3 

Eng.  Dev. 

Cutler  Hanmer/AIL 

7. 

F-16  Radar3 

Eng.  Dev. 

Westinghouse0 

Hughes 

8. 

F-16  Inertial  Navigation  Sys. 

Eng.  Dev. 

Singer-Kearfott 

9. 

F-16  Fire  Cont.  Computer 

Eng.  Dev. 

Del  co 

10. 

F-16  Flight  Cont.  Computer 

Eng.  Dev. 

Lear-Siegler 

11. 

F-16  Radar/E-0  Displayd 

Eng.  Dev. 

Kaiser  Aerospace 

11a. 

F-16  Radar/E-0  Dis.  Elect. d 

Eng.  Dev. 

Kaiser  Aerospace 

Notes : 


aThese  subsystems  were  previously  studied  by  IDA  as  Design- to- Cost 
subsystems. 

bThe  ARC- 164  UHF  radio  did  not  have  an  explicit  RIW  option  during 
development.  However,  the  development  contractors  knew  that  the  Air 
Force  was  going  to  select  the  winner  of  the  competition  based  upon 
life-cycle  cost  (LCC)  and  that  the  production  contractor  was  required 
to  guarantee  (with  profit-loss  potential)  the  achievement  of  field 
reliability  targets. 

cSuccessful  bidder  for  production  award. 

dThese  two  subsystems  were  combined  during  the  study  for  purpose  of 
analyses. 


Figure  S-l.  CANDIDATE  SUBSYSTEMS  FOR  WARRANTY  IMPACT  STUDY 
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that  would  improve  the  warranty  experiences  of  both  the  Services 
and  equipment  contractors  were  also  an  expected  output  of  the 
study . 

The  research  methodology  consisted  of  data-gathering 
through  in-depth  interviews  with  Government  and  contractor 
management  personnel  and  the  preparation,  execution,  and 
retrieval  of  a comprehensive  questionnaire.  The  questionnaire 
responses  were  then  combined  and  analyzed  as  an  aggregate  con- 
tractor sample  response.  The  interviews  were  directed  to  ac- 
quiring insights  into  the  issues  surrounding  the  questionnaire 
responses  and  also  to  identifying  peripheral  areas  that  were 
important  influences  on  contractor  behavior. 

The  scope  of  the  study  is  limited  by  the  selection  of  the 
eleven  electronics  subsystems  containing  contract  warranty 
provisions , the  time  span  of  the  research  efforty  and  the 
necessity  to  protect  competition-sensitive  and  contractor- 
proprietary  data. 

B.  PRINCIPAL  FINDINGS 

The  research  effort  yielded  a number  of  significant  find- 
ings related  to  the  application  and  impact  of  warranties  on 
the  candidate  programs. 

1 . Warranty  Requirements  and  Their  Application 

Our  investigations  into  warranty  requirements  and  their 
application  provided  findings  in  areas  of  contract  requirements, 
the  contractor  development  environment,  equipment  maturity, 
and  warranty  negotiation. 

Most  of  the  programs  investigated  contained  multiple 
warranty  obligations  or  options  geared  to  the  achievement  of 
specified  operational  reliability.  The  requirement  details 
were  similar  for  most  of  the  programs  examined. 


Equipment  warranties  covering  operational  performance  for 
time  periods  exceeding  two  years  were  found  to  be  new  contract 
requirements  for  the  candidate  contractors,  most  of  whom  lacked 
previous  military  warranty  experience.  The  warranties  therefore 
represented  a completely  new  contractual  commitment  for  the 
contractors . 

Contractors  were  not  usually  able  to  negotiate  the  detailed 
terms  and  conditions  of  their  warranty  obligation  until  negotia- 
tion of  the  production  contract.  This  situation  was  found  to 
inhibit  important  design  trade-offs  during  the  engineering 
development  program. 

Most  (eleven  of  sixteen)  of  the  contractors  described  their 
equipment  as  a "new  technological  development"  rather  than  a 
product  improvement  over  a previous  model.  The  risk  associated 
with  predicting  the  ultimate  field  reliability  of  the  equipment 
was  found  to  be  increased  because  of  the  lack  of  relevant  field 
data  on  previous  equipment. 

Difficulties  or  critical  areas  during  warranty  requirements 
negotiation  were  reported  in  eleven  major  categories.  Measure- 
ment of  field  reliability,  definition  of  relevant  failures,  and 
warranty  price  were  the  critical  negotiating  areas  most  frequently 
identified  by  the  contractors. 

2.  Impact  on  Candidate  Development  Programs 

The  overall  impact  of  warranty  requirements  on  the  candi- 
date development  programs,  in  terms  of  the  increase  in  develop- 
ment program  effort,  was  reported  to  be  small.  An  average  impact 
of  5.5  percent  increase  in  effort  was  recorded  based  on  all  six- 
teen questionnaire  responses.  The  development  program  functional 
elements  receiving  the  greatest  impact  were  contract  management, 
program  management,  design  analysis,  data  and  reports,  and 
financial  management. 
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Major  barriers  to  additional  program  impacts  were  also 
Identified.  The  major  constraints,  as  reported  by  the  con- 
tractors, were  the  uncertainty  of  warranty  requirements,  the 
shortage  of  additional  development  funds,  the  constrained 
development  program  schedule,  preset  unit-production  cost 
goals,  and  inflexible  equipment  performance  and  product  . 
specifications. 

3.  Impact  on  Product  Designs 

The  impact  of  warranty  options  on  equipment  design  was 
reported  to  be  minor,  except  for  contract-imposed  requirements 
for  box  seals,  elapsed-time  indicators,  and  warranty  labels. 

A small  impact  was  reported  in  the  areas  of  built-in  test 
circuitry  or  other  means  of  failure  isolation,  such  as  test 
equipment  accommodation. 

The  primary  barrier  to  design  impacts  appeared  to  be  the 
requirement  to  design  equipment  suitable  for  both  Service 
organic  maintenance  and  contractor  maintenance. 


Potential  Impact  on  Candidate  Development  Programs 


Three  warranty  application  conditions  that  could  lead  to 
increased  warranty  impact  were  identified  and  investigated 
during  the  research.  These  conditions  were:  (1)  negotiation 

of  warranty  terms  and  conditions  at  the  beginning  of  engineer' 
ing  development,  (2)  elimination  of  the  option  status  of  war- 
ranty contract  requirements,  and  (3)  greater  specification 
flexibility. 

The  responses  to  the  research  questionnaire  indicated 
•that  the  overall  development  program  effort  would  have  been 
increased  from  5.5  percent  to  8.2  percent  if  the  negotiation 
of  warranty  terms  and  conditions  had  been  accomplished  prior 
to  the  beginning  of  engineering  development.  The  greatest 


impact  was  recorded  in  program  management  (10. 6$),  financial 
management  (10.350,  and  contract  management  (9.450.  Similarly, 
the  program  effort  would  have  been  increased  to  10.5  percent  if 
the  warranty  terms  and  conditions  were  firm  rather  than  an  option 
at  the  beginning  of  engineering  development.  The  greatest  impact 
was  recorded  in  program  management  (14$),  data  and  reports  (12$), 
and  materials  and  purchased  parts  (11.8$).  If  greater  specifi- 
cation relief  were  additionally  permitted,  the  entire  system 
would  readjust  so  that  the  total  development  effort  (compared 
with  the  actual  case)  would  increase  only  nominally  to  5.6  per- 
cent. However,  the  areas  of  increased  effort  were  now  reported 
to  be  analyses  (9-3%),  data  and  reports  (8.0$),  and  financial 
management  (7.8$). 

5.  Correlation  of  Contractor  Responses 

Certain  contractors  consistently  reported  greater  increases 
in  development  program  effort  than  did  other  contractors  over 
all  conditions  examined.  For  those  contractors  reporting  a 
warranty  impact  other  than  zero,  the  estimated  impact  on  the 
candidate  development  program  averaged  12.4  percent.  The 
reported  impacts  for  the  beginning , firmy  and  specification 
relief  conditions  were  12.8,  15.1,  and  9-9  percent,  respectively. 
Except  for  the  actual  warranty  application  case,  this  contractor 
subset  reported  an  average  impact  that  was  60  percent  higher 
than  the  total  aggregate  sample. 

Attempts  to  correlate  this  contractor  subset  with  other 
subsets  were  unsuccessful.  Attempts  to  identify  relationships 
between  contractor  response  and  program  status.  Service  sponsor, 
competitive  environment,  major  system  application,  and  other 
characteristics  were  also  unsuccessful.  No  identifiable 
rationale  for  specific  contractor  behavior  could  be  found. 
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6.  Additional  Issues  Relating  to  Warranty  Impacts 

The  following  additional  issues  relating  to  warranty  impacts 
were  identified  during  the  research  effort: 


• Contractor  Motivation  for  Increased  Reliability 

• Government  Credibility  for  Warranty  Success 

• Competitive  Environment 

• Prediction  of  Field  Reliability 

• Differences  Between  Prototype  and  Production  Equipment 

• Design-to-Cost  vs.  Design- for-Warranty 

• Technological  Innovation  vs.  Design  Conservatism 

• Return-on-Investment  Considerations 

• Cost  Impact  of  Equipment  Warranties 

• Subsystem  Contracting  Environment 

• Legal  Enforceability  of  Warranty  Contracts 


These  issues,  either  individually  or  in  the  aggregate, 
were  reported  to  influence  contractor  behavior  during  engineer' 
ing  development  by  helping  to  create  a greater  impact  or  by 
presenting  barriers  that  precluded  further  impact. 


B.  CONCLUSIONS 


Based  on  the  findings  of  the  limited  research  effort,  the 
following  conclusions  can  be  drawn  regarding  warranty  impacts 
on  the  candidate  subsystems  examined  during  the  study. 


1.  Warranty  Application  Has  Been  Uncertain  and  Imperfect 


The  application  of  warranties  to  the  candidate  electronics 
subsystem  development  programs  was  made  In  an  environment  of 
uncertainty  created  by  the  experimental  nature  of  the  warranty 
acquisition  policy,  the  lack  of  previous  contractor  experience 
with  warranties,  the  advanced  technological  status  of  the 
equipment,  and  the  lack  of  definitized  warranty  requirements 
during  development.  This  environment  and  contractor  uncertainty 
about  the  Government's  ultimate  exercise  of  the  warranty  option 
resulted  in  an  imperfect  application  of  warranty  techniques. 
Because  of  these  conditions,  we  believe  the  warranty  experi- 
ments represented  by  the  candidate  subsystems  may  not  be 
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conclusive  indicators  of  the  success  or  failure  of  the  Govern- 
ment's warranty  policy. 

2.  Warranty  Options  Are  Not  Yielding  Design  and  Development 

Incentives  for  Improved  Reliability 

Given  the  uncertainties  surrounding  the  application  of  war 
ranties  and  the  observed  impacts  on  the  candidate  programs,  it 
is  concluded  that  warranty  option:,  are  not  changing  the  equip- 
ment design  and  development  process  in  ways  that  would  either 
significantly  improve  the  inherent  reliability  of  the  equip- 
ment or  provide  necessary  additional  information  to  contractors 
regarding  expectations  of  future  equipment  field  reliability. 
Presently,  the  candidate  warranty  options  can  be  better 
characterized  as  similar  to  fixed-price  options  for  future 
maintenance  of  production  equipment. 

3.  The  Potential  Exists  for  Increased  Warranty  Impact 

There  is  a potential  for  increased  beneficial  warranty 
impact  on  the  equipment  design  and  the  development  process  if 
uncertainties  and  barriers  surrounding  warranty  application 
are  removed.  Elimination  of  the  optional  nature  of  present 
warranty  programs,  for  example,  appears  to  be  an  important 
step.  The  increased  impact  can  be  achieved  most  efficiently 
if  maximum  allowable  specification  relief  accompanies  changes 
in  warranty  policy.  Realization  of  the  increased  impact  could 
necessitate  longer  development  schedules  and  additional  develop 
ment  funds  because  of  the  increased  effort  involved. 

4.  Warranty  Options  Have  Captured  Management  Attention 

The  interviews  and  questionnaire  responses  demonstrated 
that  warranty  options  have  attracted  the  attention  and  concern 
of  Service  and  contractor  management.  This  concern  has  been 
prompted  by  the  great  uncertainties  surrounding  the  candidate 


program  outcomes.  Steps  to  translate  this  concern  into  posi- 
tive action  for  improved  equipment  reliability  are  potentially 
available  (see  below). 

0.  GUIDELINES  FOR  WARRANTY  POLICY  DEVELOPMENT  AND 

APPLICATION 

The  findings  derived  from  Government  and  contractor  inter- 
views and  the  data  retrieved  during  this  research  study  support 
several  guidelines  for  continued  warranty  policy  development. 

1 . Optimal  Scheduling  for  Warranty  Decisions 
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In  order  to  achieve  maximum  benefits  from  equipment  war- 
ranties, the  maintenance  philosophy  for  the  candidate  equip- 
ment should  be  established  by  the  completion  of  the  equipment's 
advanced  development  stage,  i.e.,  before  DSARC  II.  The  option 
process  for  exercising  warranties  should  be  discarded  as 
counterproductive  to  optimizing  equipment  design  and  to 
achieving  increased  equipment  reliability  and  maintainability 
through  the  development  process. 

2 . Warranty  Requirements  Definition 

The  specifications  and  requirements  for  future  equipment 
warranties  should  be  negotiated  early  in  the  equipment  develop- 
ment program,  preferably  during  negotiation  of  the  engineering 
development  contract.  These  requirements  should  be  treated  the 
same  as  equipment  design  or  performance  specifications. 

3.  Integration  of  Warranties  with  Design- to-Cost  Programs 

The  application  of  warranties  to  programs  that  already  con- 
tain design-to-cost  goals  should  initiate  a reassessment  of 
the  production  cost  targets  to  minimize  the  sum  of  future  acqui- 
sition and  operational  costs.  A formal  process  should  be 
established  to  enable  contractors  and  Service  project  officers 
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to  revise  unit-production  cost  goals  if  such  a change  can  be 
justified  on  the  basis  of  reduced  life-cycle  costs. 


Specification  flexibility  for  non-critical  requirements 
assumes  a role  of  even  greater  importance  when  contractors 
are  required  to  assume  additional  responsibilities  for  equip 
ment  performance.  Thus,  specification  flexibility  should  be 
an  integral  part  of  each  warranty  procurement. 


Reliability  Data  and  Predictive  Models 


To  aid  in  improving  reliability  prediction,  retrieval  of 
current  and  past  field  reliability  data  on  all  generic  equip- 
ment suitable  for  warranty  application  should  be  continued. 
Efforts  also  should  be  increased  to  predict  more  accurately 
future  field  reliability  from  demonstration  test  data.  These 
initiatives  should  be  taken  with  help  from,  and  feedback  to, 
the  industrial  contractor  sector. 


Development  Funding  and  Schedule  Flexibilit 


Preparations  for  meeting  equipment  warranty  guarantees  may 
require  additional  development  time  and  funds.  The  Defense 
Department  should  recognize  this  possibility  and  take  steps  to 
ensure  that  additional  time  and  money  are  available. 
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INTRODUCTION 


A.  BACKGROUND 

During  the  past  decade,  the  Department  of  Defense  has  been 
concerned  with  ways  to  increase  the  availability  of  operational 
equipment  and  to  decrease  the  costs  associated  with  operations 
and  support.1  This  concern  has  been  focused  on  those  systems 
and  subsystems  that  most  often  lead  to  poor  availability  because 
of  deficiencies  in  reliability  or  maintainability.  In  many  in- 
stances, electronics  subsystems  proved  to  be  the  critical  ele- 
ments of  complex  systems  that  restricted  system  reliability, 
maintainability,  and  ultimately,  system  availability.  Efforts 
to  improve  field  experience  with  electronics  systems  and  subsys- 
tems have  included  equipment  improvement  and  retrofit  programs, 
greater  emphasis  upon  fault  isolation  and  built-in  test  capability, 
integrated  test  facilities,  improved  maintenance  training,  and 
contractor  guarantees  or  warranties  for  equipment  field 
performance . 

Contractor  equipment  warranties  covering  equipment  operation 
over  periods  longer  than  one  year  were  initiated  within  the 
Defense  Department  by  the  Navy  in  1967  with  the  negotiation  of 
five-year  "failure-free"  warranties  for  navigational  gyro  plat- 
forms produced  by  the  Lear  Siegler  Corporation.  Although  the 
Lear  Siegler  warranty  covered  equipment  that  already  had  accrued 


^Availability  is  the  primary  measure  of  subsystem  utility  and  effectiveness 
based  upon  the  assumption  that  "available"  equipment  is  operating  in  accord- 
ance with  all  design  and  performance  specification  values,  including  reli- 
ability. 


field  operating  experience  and  had  an  established  operational 
reliability  history,  subsequent  Navy  contracts  for  aircraft  altim- 
eters (Honeywell  APN-194)  and  radio  receivers  (Northrop  Omega) 
extended  the  failure-free  warranty  concept  to  provide  coverage 
from  initial  equipment  deployment  up  to  a two-year  operational 
period.  In  1973 > the  possibility  of  extended-year  warranty 
provisions  was  introduced  into  several  Service  electronics 
development  plans  in  response  to  a request  by  the  Office  of  the 
Secretary  of  Defense  that  trial  use  of  warranties  be  introduced 
in  the  acquisition  and  initial  operation  of  a number  of  candi- 
date electronics  subsystems.1  This  action  was  motivated,  in 
part,  by  the  commercial  successes  achieved  by  the  airlines  with 
equipment  warranties  and  by  the  successful  results  of  the  early 
Navy  applications. 

Since  1973>  the  trial  application  of  extended-term  equip- 
ment warranties  has  been  encouraged  by  subsequent  OSD  memoran- 
dums that  have  clarified  the  concept,  formulated  application 
guidelines,  and  established  working  groups  within  OSD  and  the 
Services  to  develop  improved  policies  and  procedures  for  moti- 
vating contractor  performance  under  warranty  contract  provisions. 

During  this  period,  several  studies  were  made  of  the  conceptual 
basis  for  warranty  use,  the  details  of  warranty  provisions,  and 
the  experiences  of  both  the  Government  and  its  equipment  con- 
tractors during  the  negotiation  of  production  contracts  contain- 
ing warranties.3 

JOSD  Memorandum  to  the  Service  Secretaries,  Trial  Use  of  Warranties  in  the 
Acquisition  Process  of  Electronics  Subsystems , 17  August  1973. 

2ASD(I&L)  Memorandum  to  the  Service  Assistant  Secretaries,  Trial  Use  of  Reli- 
ability Improvement  Warranties  in  the  Acquisition  Process  of  Electronic  Sys- 
tems/Equipments  - Action  Memorandum,  14  August  1974.  ASD(I&L)  Memorandum  to 
Assistant  Secretaries,  same  subject  as  preceding  memo,  1 November  1974.  OSD 
Memorandum  to  the  Assistant  Secretaries  of  the  Military  Departments  (R&D), 

(I&L),  Tri-Service  Reliability  and  Support  Incentives  Group , 5 September  1975. 

ASD(I&L)  Memorandum  to  the  Assistant  Secretaries  of  the  Services,  Reliability 
Improvement  Warranty  Guidelines,  16  September  1975- 

3The  reader  is  referred  to  Appendix  P,  "Annotated  Bibliography,"  for  more  in- 
formation. 
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In  compliance  with  the  OSD  directives,  the  Services  subse- 
quently Introduced  options  for  warranties  or  maintenance  guar- 
antees into  a number  of  electronics  subsystems  currently  in  de- 
velopment. The  subsystems  selected  were  initially  limited  to 
those  that  had  been  designated  as  experimental  design-to-cost 
acquisitions.  As  warranty  option  contract  clauses  and  require- 
ments were  developed,  however,  the  application  of  warranties 
was  increased.  The  most  recent  application  of  warranty  options 
was  for  the  major  avionics  equipment  being  developed  for  the  Air 
Force  F-16  air  combat  fighter  aircraft. 

Two  studies  by  the  Institute  for  Defense  Analyses  addressed 
warranties  for  electronics  subsystem  acquisitions  as  part  of  a 
broader  study  of  ways  to  reducTe  electronics  costs  and  improve 
electronics  reliability.1  'The  1974  Electronics-X  study  found 
that  equipment  warranties  could  provide  strong  contractor  moti- 
vation to  design  equipment  for  increased  reliability  through 
the  internalization  of  field  reliability  and  maintenance  costs. 
The  subsequent  Design-to-Cost  (DTC)  study  found  a potential  for 
significant  equipment  design  and  development  program  impacts 
resulting  from  the  early  introduction  of  warranties  into  candi- 
date DTC  programs.  A study  recommendation  for  a detailed  exami- 
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nation  of  the  impact  of  warranty  options  on  equipment  design  and 
the  engineering  development  program  resulted  in  the  present 
research  effort.  ✓ 

B.  RATIONALE  AND  APPROACH 

Based  upon  the  previous  IDA  study  of  DTC  programs  contain- 
ing reliability  Improvement  warranty  (RIW)  requirements,  there 
appear  to  be  a number  of  activities  or  phases  in  the  equipment 


toward  P.  Gates,  Jr.,  etal.,  Electronics -X:  A Study  of  Military  Electron- 

ics with  Particular  Reference  to  Cost  and  Reliability , IDA  Report  R-195 
(1974);  and  C.  David  Weimer,  The  Application  of  Design-to-Cost  Acquisition 
Policies  to  Selected  Electronics  Subsystem  Development  Programs , IDA  Study 

S-459  (1975). 
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design  and  development  process  that  could  be  significantly  af- 
fected by  the  imposition  or  expectation  of  future  contractor 
maintenance  responsibilities.  Before  investigating  these  areas, 
however,  it  is  necessary  to  examine  the  premise  that  actions 
leading  to  greater  equipment  reliability  can  be  identified  in 
the  design  and  development  process. 

The  primary  motivations  for  establishing  warranty  contract- 
ing arrangements  are  to  increase  equipment  mission  availability, 
through  increased  field  reliability,  and  to  decrease  equipment 
maintenance  costs,  through  both  increased  field  reliability  and 
improved  equipment  maintainability.  Therefore,  an  assessment 
of  warranties  as  a procurement  tool,  must  also  examine  the  motiva- 
tions and  opportunities  available  for  achieving  these  objectives.1 


1 . Methods  for  Achieving  Reliability  and  Maintainability 

Goal  s 

A review  of  the  literature  and  DoD  policy  directives  and 
memorandums  reveals  that  the  objectives  of  increased  equipment 
reliability  and  improved  maintainability  can  be  achieved  during 
two  major  phases  of  an  equipment  life  cycle.  First,  increased 
reliability  (and  maintainability)  can  be  achieved  through  greater 
effort  during  design  and  development.  This  has  been  the  demon- 
strated approach  to  reliability  for  both  the  NASA  and  military 
space  programs,  in  both  of  which  equipment  cannot  be  repaired  or 
maintained  after  deployment.2  The  second  opportunity  to  improve 
reliability  and  maintainability  occurs  in  the  operational  phase, 
after  equipment  development  has  been  completed.  Based  on  opera- 
tional experience,  the  equipment  can  be  redesigned  and  new  com- 
ponents can  be  retrofitted  in  the  field  to  "fix"  reliability 


1 These  objectives  are  assumed  to  be  subject  to  trade-offs  within  the  overall 
goal  of  reducing  equipment  total  life-cycle  costs. 

2See  for  example.  Air  Force  Studies  Board,  National  Research  Council,  Nation- 
al Academy  of  Sciences,  "Reliability  in  Aeronautical  Avionics  Equipment," 
(Washington,  D.C.,  1975). 
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problems  uncovered  during  operational  use.  This  approach  is 
the  one  that  has  traditionally  been  followed,  particularly  with 
electronics  subsystems.  But  unfortunately,  this  method  has 
proven  to  be  extremely  costly  and  has  seldom  restored  reliability 
to  acceptable  levels.1 

Both  approaches  to  improved  reliability  involve  a design 
effort,  whether  initially  or  in  a redesign  of  production  config- 
uration equipment.  And  both  approaches  require  proving  the  de- 
sign through  a test-fix-test  process.  These  actions  constitute 
the  heart  of  the  initial  reliability  development  and  demonstra- 
tion program,  as  well  as  any  subsequent  product  improvement 
program. 

2 . Contractor  Motivations  for  Improved  Reliability 

During  Design  and  Development 

The  rationale  presented  above  can  now  be  applied  to  the 
conditions  imposed  upon  a contractor  who  is  developing  equipment 
that  will  (or  at  the  Government's  option,  may)  be  covered  by  a 
warranty  during  the  first  several  years  of  field  deployment. 

Equipment  warranties  have  been  developed  in  an  attempt  to 
internalize  the  problem  of  degraded  field  reliability  to  the 
equipment  manufacturer  such  that  he  will  have  a strong  incentive 
to  minimize  reliability  problems  in  operational  equipment  and 
respond  effectively  to  any  problems  that  become  evident  as  equip- 
ment is  placed  in  use.  The  incentive  mechanism  is  contractor 
financial  profit  or  loss,  depending  upon  the  difference  between 
the  measured  performance  of  the  equipment  and  the  predicted  per- 
formance used  to  establish  the  price  of  the  warranty  agreements. 


^e  Delco  Carousel  inertial  navigation  subsystem  is  a notable  exception 
to  this  statement.  General  Motors  invested  several  millions  of  its 
own  money  but  did  successfully  improve  field  reliability  under  airline 
conditions  to  meet  the  warranty  PfTBF  goal  of  1000  hours. 
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The  price  of  the  warranty  will  be  negotiated,  based  upon 
the  expected  or  specified  field  reliability.1  The  contractor, 
accordingly,  must  take  those  actions  during  design  and  engineer- 
ing development  that  will  reasonably  assure  that  the  specified 
field  reliability  will  be  achieved  and  that  the  warranty  will 
be  a profitable  part  of  the  overall  equipment  development  pro- 
gram . 


Emphasizing  the  design  process  as  the  primary  method  of 
achieving  improved  reliability  under  conditions  of  warranty  ob- 
ligation has  been  consistently  recognized  by  DoD  organizations. 
Evidence  of  this  recognition  is  contained  in  the  DoD  statements 
quoted  in  Figure  1. 


Date 

Organization 

Key  Statements 

14  Aug  73 

OSD  (Director  of 
Defense  Research 
& Engineering) 

"Many  techniques  (to  achieve  greater 
field  reliability)  have  been  employed 
...  i.e.,  more  ...  reliable  products 
are  designed " 

24  July  74 

USAF(Headquarters) 

"Contractors  will  be  motivated  to 
ensure  that  their  equipment's  relia- 
bility and  maintainability  are  given 
appropriate  attention  at  the  time  it 
is  initially  designed,  since  this 
could  affect  its  subsequent  repair  or 
replacement  costs." 

14  Aug  74 

OSD  (Installations 
& Logistics) 

"The  objective  of  a RIW  is  to  motivate 
and  provide  an  incentive  to  contrac- 
tors to  design  and  produce  equipment 
which  will  have  low  failure-rates  and 
low  repair  costs " 

11  June  76 

OSD  (Joint  Logis- 
tics Commanders) 

"The  use  of  RIWs  on  selected  hardware 
can  provide  additional  motivation  to 
the  contractors  to  consider  opera- 
tional supportabil ity  factors  during 
the  design  and  development  process. 

Figure  1.  EVIDENCE  OF  WARRANTY  IMPACT  ON 
DESIGN  AND  DEVELOPMENT 


*If  all  goes  according  to  predictions,  the  warranty  becomes  a maintenance 
contract  with  fee  or  profit  incentives  to  improve  reliability  and  reduce 
maintenance  costs  during  its  tern. 
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Another  closely  related  problem  that  must  be  addressed  in 
the  design  and  development  process  is  the  inability  to  accurately 
predict  ultimate  field  reliability  from  development  test  results. 
The  Defense  Department  recognized  this  particular  problem  in  a 
recent  memorandum  that  emphasized  that  field  reliability,  support 
costs,  and  potential  for  reliability  growth  should  be  "reasonably 
predictable"  prior  to  RIW  price  bid.2  The  Council  of  Defense  and 
Space  Industry  Associations  (CODSIA)  responded  to  this  RIW  imple- 
mentation directive  by  suggesting  that  the  "risk  of  reasonable 
predictability  . . . can  be  satisfactorily  mitigated  only  by  pro- 
viding for  increased  development  and  environment  testing...."3 

Improved  field  reliability  and  maintainability,  therefore, 
are  perceived  by  both  the  Government  and  industry  as  requiring 
design  and  development  attention  either  during  engineering  de- 
velopment or  after  operational  experience  in  the  field.  And 
since  the  terms  and  conditions  of  warranty  contract  agreements 
can  include  severe  cost  penalties4  for  unsatisfactory  reliability 
after  field  deployment,  the  achievement  of  "satisfactory"  reli- 
ability must  necessarily  be  accomplished  in  the  development  stage 
It  Is  then  that  efforts  must  be  taken  to  answer  the  two  ques- 
tions: How  can  the  specified  reliability  be  achieved  and  demon- 

strated through  design  and  test?  What  will  the  ultimate  reli- 
ability be  in  the  field  with  production-type  equipment? 


Perhaps  the  problem  of  predictability  is  the  primary  one  to  be  solved.  The 
question  is  whether  the  predicted  reliability  is  an  accurate  estimate  of 
ultimate  field  reliability. 

2ASD(I&L)  Memorandum  for  the  Service  Secretaries,  Reliability  Improvement 
Warranty  Guidelines,  16  September  1975* 

3 CODSIA  Position  Paper,  "Recommendations  on  Reliability  Improvement  Warranty 
(RIW),"  RIW  Task  Group,  Case  19-5,  30  December  1975. 

4Ihe  tern  "penalty"  is  used  generically  throughout  this  report  to  describe 
reduction  in  contractor  profit  or  an  increase  in  nonreimbursable  costs 
resulting  from  deficiencies  in  equipment  or  contractor  performance. 
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3 . Impacts  Resulting  from  Warranties 

Changes  in  the  equipment  design  configuration  as  a result 
of  warranty  obligations  can  be  expected  for  several  reasons,  in 
addition  to  improving  the  inherent  system  reliability.  The  very 
nature  of  the  warranty  option  will  dictate  design  provisions. 
Secure  equipment  seals,  for  example,  may  be  needed  to  protect 
against  unauthorized  entry  or  repair.  Elapsed  operating  time 
indicators  may  also  be  installed  to  monitor  the  reliability 
operating  time  interval  on  a replaceable  unit.  Other  changes 
in  design  are  possible  to  improve  maintainability,  such  as  in- 
cluding built-in  test  circuitry  and  simplifying  the  design  to 
permit  easy  repair  or  replacement  of  component  parts. 

In  a similar  way,  changes  in  the  "standard"  development 
program  can  be  anticipated  to  better  predict  field  maintain- 
ability and  reliability.1  More  design  analysis  should  be  ex- 
pected, additional  environmental  testing  could  be  necessary,  and 
detailed  analyses  involving  prediction  of  reliability  and  future 
anticipated  maintenance  experience  might  be  required. 

Thus  far,  it  might  be  assumed  that  a warranty  impact  can  be 
anticipated  only  if  a new  equipment  design  is  to  be  developed. 
The  potential  for  warranty  impact  on  a design  and  development 
program  is  not  limited  to  new  equipment  designs,  however;  it 
exists  whenever  any  one  of  several  conditions  is  present.  Some 
of  the  criteria  sufficient  for  an  impact  resulting  from  the 
ultimate  requirement  to  guarantee  equipment  maintenance  cost 
and  field  reliability  are  listed  below: 

(1)  A new  equipment  design  is  to  be  developed. 

(2)  "Off-the-shelf"  equipment  does  not  presently  meet 
reliability  goals  or  specifications. 


1 There  is,  of  course,  no  "standard"  electronics  subsystem  development  pro- 
gram format.  But  the  ingredients  of  a development  program,  i.e.,  design 
reviews,  failure  analyses,  and  reliability  demonstration  test  programs,  are 
heavily  regulated  by  Government  specifications  and  Service  program  planning 
documents . 
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(3)  Necessary  modifications  to  existing  equipment  will 
affect  reliability  or  maintainability. 

(4)  Proposed  equipment  must  be  designed  for  either 
organic  or  contractor  warranty  maintenance. 

(5)  Operating  environment  has  changed  (increased 
severity)  for  the  proposed  equipment. 

(6)  Contractor  has  little  or  no  previous  long-term 
product  warranty  or  support  experience. 

C.  STUDY  OBJECTIVES  AND  SCOPE 

The  objective  of  this  study  was  to  investigate,  record,  and 
analyze  the  impact  of  warranty  options  upon  the  design  and 
development  process  of  selected  candidate  electronics  subsystems. 
Detailed  objectives  of  the  study  were  as  follows: 

(1)  Document  actual  impacts  observed  for  candidate 
programs . 

(2)  Identify  areas  of  potential  RIW  impact. 

(3)  Identify  barriers  to  achieving  beneficial  impacts. 

(4)  Identify  dysfunctional  elements  associated  with 
RIW  application. 

(5)  Identify  actions  that  OSD  and  the  Services  should 
consider  to  improve  the  application  and  implementa- 
tion of  RIWs . 1 

Because  the  earlier  DTC  study  at  IDA  had  developed  infor- 
mation on  a number  of  electronics  subsystems,  seven  of  those 
subsystems  were  selected  for  analysis  in  this  study.2  In  addi- 
tion, four  other  candidate  electronics  subsystems  being  developed 
with  warranty  option  provisions  were  selected  for  study.  These 
subsystems  were  all  avionics  being  developed  for  the  Air  Force 


1 These  objectives  are  as  stated  in  the  IDA  Task  Order,  Appendix  A. 

2Weimer,  The  Application  of  Design  to  Cost  Acquisition  Policies  to  Selected 
Electronics  Subsystem  Development  Programs. 
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P-16  fighter  aircraft  under  subcontract  to  General  Dynamics  Cor- 
poration.1 A total  of  eleven  candidate  subsystems,  involving 
seventeen  contractor  organizations  during  development,  were  thus 
selected  for  study.  These  subsystems  and  contractors  are  listed 
in  Figure  2. 2 


The  scope  of  the  study  was  further  defined  in  terms  of 
research  concentration  after  preliminary  discussions  with  both 
the  OSD  project  office3  and  the  IDA  Technical  Review  Board. 

Five  major  areas,  discussed  below,  were  selected  for  investiga- 
tion and  analysis. 


Contract  Requirements.  The  first  area  of  investigation 
was  the  requirements  for  product  warranties  that  were 
known  or  imposed  during  the  development  program.  An 
analysis  of  warranty  impacts  could  not  be  undertaken 
without  an  understanding  of  the  contractual  require- 
ments that  were  forcing  the  actions. 


Contractor  Warranty  Background.  Another  variable  that 
plays  an  important  role  in  the  contractor  response  is 
past  exposure  to  and  experience  with  warranties.  Impacts 
were  believed  to  be  more  difficult  to  identify  if  the 
contractor  had  already  incorporated  warranty  guidelines 
or  procedures  into  his  development  process. 


Development  Program  Impact.  This  was  the  major  area  of 
research.  Warranty  impacts  identified  in  this  area  would 
be  important  indicators  of  actions  contractors  con- 
sidered necessary  to  attain  better  reliability  as  well 
as  to  reduce  risk.  The  results  of  this  research  effort 
could  possibly  determine  the  direction  of  future  pro- 
gram planning  and  policy  application. 


lrThe  basic  General  Dynamics  (GD)  contract  with  the  Government  contains 
warranty  option  provisions  for  specific  avionics  subsystems . These  pro- 
visions were  passed  down  to  subsystem  contrators  as  part  of  the  GD 
purchase  order  in  essentially  the  same  form  as  negotiated  by  GD. 


2 An  additional  subsystem,  the  F-16  heads-up  display,  was  considered  for  in- 
vestigation as  part  of  the  F-16  avionics  warranty  program.  However,  sub- 
sequent discussions  with  the  F-16  project  office  revealed  that  data  ac- 
quistion  would  involve  European  travel.  It  was  decided  that  the  time  and 
cost  involved  did  not  justify  retaining  this  program  as  part  of  the  sub- 
system sample. 


3OSD,  ODDR&E  (Planning). 


10 


* 


J 


! 


• • r—  H— 

c n • • 10  •*- 

C • . H-  4-  O r— 

. Z1*-  f • ip 

— -O  X •*-  *-  r-  « ~ . . o 

• IOC*  *—  • *0  <o  <o  • -C  5 * 

«rtO  M W . • iO  V)U  U k J VI  f • VI 

•a  *00  t/»  O r—  <0  <0  >*  r— 

■r“  • "f  • *»—  »—  • • .O  • 3 » r— 

0.0  0)  O.Z  •—  • IO  o C 1-  J*  0)  •-  «o 

ig  o>  cio  o a;  u.  oi  o « 4-  • 4-  o u_ 

CC  0)  >»«  * Q.+J  0)  «*j  cfi  IO  4>  O 

•i-  IO  C IO  C 0)  **-  L-  0.  p-  g U Of 

4-0  3 4-  0)  0)  Q r-  Q o IO  4->  -r-  v r- 

IO  IO  -Q  CX  «M  r-  4J  L.  4-»  *J>  +> 

O C * TJ  £ C 4-»  C r-  C OMO  r>  p-  «J 


• 3 -S  C 

< o •-  io 


4J  t*  0)  r- 
WP  P > 4J 
0>  IO  p-  r-  +> 


O C *wt=  C V C r-  C V « r-f- 

Olio  P 0>  <0  f-1—  *r—  IO  310  0)0)  103  •*- 

oco  u.  o o xuj  — j co  u.  co  a on  coo  _j 


o •«  • 

> 4>  4-» 

5§tt 

-55 

U f-  •*- 

l 

> Ow 


r—  ^ o)  c 

0)  C I c o 

x io  4-  >>  m 4) 

>>  E <d  -a  a;  _c 

SO)  4-  cn  0)  .C  P 

C 4-  C r—  C7»  >, 

o o •*-  a>  3 io 

CC  XX  00  h-  X oc 


*-•  a> 

< a p 

IO  4-> 

4-  a.  o o 

I8S  t 

E 4-  3 IO 

<0  0)0  CD 

X «t  _c 

a>  I 

4.  cn  C (/I  4. 

a>  c -r-  o>  a> 
*T  T **  -C  OD 
O M o C 


P O)  t/> 

3 0 0) 

O CO  3 


Q Q 

OO  00 

UJ  C 


OD  "O  XT 

ro  io  o o 

\ P 4-  4- 

vo  co  a.  a. 


s 

§ 

UJ 

> 

Q 

Q 

Q 

(3 

Q 

Q 

O (/) 

o 

UJ 

o 

UJ 

2 

O0 

< 

CO 

< 

vo 

< 

OO 

< 

s <3 

Dev 

Dev 

Dev 

Dev 

> 

> 

£ 

> 

£ 

> 

£ 

> 

£ 

Eng. 

cn 

c 

UJ 

cn 

c 

UJ 

Eng. 

S' 

UJ 

Eng. 

cn 

c 

UJ 

Eng. 

Eng. 

P P <0 

h! 

O Li) 

f-  CJ 

a)  co 

> C 0) 
4-r  L 
*o  J f- 

0»  «J  O' 

c x <D 

•r-  P 4. 

4. 

3 P V> 

■O  O 10 

c * * 

O « L • 
•r  Ifl  O 
P P P 
OOU4I 

0 P #o  cn 

4.  4. 
3 o>p  io 
►-«  C C P 
at  •*”  o 
o o >» 

P O)  P 
f-  C *»“ 

y «i  o<“ 

•P*  <0  **-  T“ 

•-  J 4Ji3 

a.  u <o 

X 01  3 -r- 
4)  OOr- 
4-0  0) 
C O 4-  4- 
iol.  a 

■o 

01  4-  0)  •— 

> *T“  -C  0) 
<0  < P •*“ 


| 

O 

4-» 

< 

< 

H- 

oc 

o> 

u. 

to 

r— 

X 

X 

§ 

CO 

vo 

§ 

TD  0) 
>>  •— 
<o  UJ 


! E £ -?3  3 3 

: 0)  *-  r-  3 ro  IO 

i +)  u_  u_  a.  ot  at 

V)  E 

> >)  VO  VO  o VO  VO 

■CO  r—  r—  O r—  r— 


• O •—  .- 

0>  r—  r— 


OiOOC 

«S| 

0 X—  > 
•r  aiu  a) 
*0  C o f“ 

4-  

IA  IO 
LC  L4J 
X Q (/>  0) 
13  o x: 

U u 4-» 
to 

VO  4-  0)  ^ 
^ *“  r- 

1 C O IQ 
O O >>f- 
O U U4J 

<vig 

0)  C 4-  4J 

£ a 


0 o 

3 O 

■O 

O « 

Q.  O 

O V) 

g 

4-  ■*-> 

0)  to 

“O  >» 

1 I 


« = I 

T-  to  ■P 


Equipment  Design  Impact.  The  area  of  equipment  design 
Impact  was  also  considered  a significant  part  of  the 
research  effort  since  specific  or  generic  design  changes 
in  preparation  for  warranty  obligations  would  be  con- 
crete evidence  that  changes  were  occurring  and  that 
equipment  design  was  reflecting  concern  about  the 
possibility  of  future  reliability  guarantees. 

Potential  Development  Impacts  and  Implementation  Barriers. 
Finally,  it  was  recognized  that  perhaps  a number  of  poten- 
tial development  impacts  could  be  identified  that  would 
help  both  the  Government  and  its  contractors  achieve 
greater  field  reliability  and  reduce  risks  as  well. 
Therefore,  potential  areas  of  RIW  impact  and  the  bar- 
riers preventing  those  impacts  from  being  exploited  were 
also  investigated. 

The  overall  research  effort  was  based  upon  the  premise  that 
contractors  with  potential  equipment  warranty  obligations  would 
undertake  efforts  to  tailor  their  equipment  design  and  develop- 
ment program  effort  to  improving  reliability  and  minimizing 
risk.  The  findings,  therefore,  will  relate  to  the  input  side 
of  the  development  process  rather  than  to  the  results  ultimately 
achieved  by  contractor  initiatives. 

The  application  of  warranties  by  the  Services  has  thus  far 
been  classified  as  "experimental,"  and  it  will  be  a number  of 
years  before  the  effect  of  any  additional  development  efforts 
can  be  assessed.  Therefore,  it  would  be  premature  at  this  time 
to  assess  the  success  or  failure  of  the  warranty  acquisition 
concept . 


D.  STUDY  METHODOLOGY 

The  study  was  conducted  in  three  successive  phases.  Phase 
I consisted  of  selecting  suitable  candidate  programs,  reviewing 
the  published  literature  in  the  field,  and  conducting  prelimi- 
nary interviews  with  both  contractor  and  Government  personnel. 
Phase  II  consisted  of  detailed  interviews  and  data-gathering 
meetings  with  contractor  personnel  who  were  responsible  for 
the  development  programs.  Phase  III  consisted  of  analysis  of 
the  data  and  report  preparation. 
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The  Phase  I effort  occupied  approximately  90  days,  during 
which  the  eleven  candidate  subsysttms  (shown  in  Figure  2)  were 
selected,  the  annotated  bibliography  (presented  in  Appendix  F) 
was  prepared,  and  initial  project  office  and  contractor  visits 
were  conducted. 

The  results  of  the  Phase  I effort  proved  critical  to  the 
conduct  and  ultimate  success  of  the  project.  It  was  found  that 
special  access  was  required  to  proceed  with  data-gathering  for 
the  F-l6  avionics  equipment.  Thus,  a major  part  of  the  Phase  I 
effort  was  spent  obtaining  the  required  OSD  and  Air  Force  en- 
dorsement of  the  study,  which  permitted  official  access  to  the 
F-16  program  office  at  the  Aeronautical  Systems  Division  (ASD), 
the  prime  contractor.  General  Dynamics  Corporation  (Ft.  Worth 
Division),  and  the  major  avionics  subcontractors.1 

A critical  output  of  the  Phase  I effort  was  the  decision 
to  structure  part  of  the  contractor  data-gathering  according  to 
a standard  research  questionnaire.  The  initial  exploratory 
interview  sessions  revealed  that  potential  warranty  impacts  can 
be  affected  significantly  by  the  timing,  certainty,  and  program 
flexibility  associated  with  warranty  application.  In  order  to 
permit  comparison  among  responses  under  several  different  con- 
tracting conditions,  a detailed  questionnaire,  included  as 
Appendix  D,  was  constructed.  This  questionnaire  was  also  used 
as  a vehicle  for  recording  information  concerning  contractor 
experience  with  warranties  and  for  acquiring  specific  case  study 
examples  of  design  and  development  program  impact.  The  major 
sections  of  the  questionnaire  (Parts  B and  C)  addressed  RIW 
impacts  as  actually  experienced,  the  potential  impacts  if  the 
RIW  option  had  been  negotiated  prior  to  engineering  development, 
and  potential  impacts  that  might  be  anticipated  if  RIW  had  been 
a firm  requirement  (rather  than  an  option)  from  the  beginning 


Copies  of  the  Air  Force  and  OSD  endorsement  memorandums  are  enclosed  as 
Appendixes  B and  C,  respectively. 
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of  the  engineering  development  phase.  An  additional  variant  was 
added  to  Part  C to  determine  the  effect  of  maximum  specification 
flexibility,  a topic  that  had  been  raised  during  the  initial  con- 
tractor meetings  as  a continuing  barrier  to  potential  impacts. 

Phase  II  consisted  of  in-depth  contractor  interviews.  These 
interviews  comprised  a formal  briefing  by  the  investigators, 
which  described  the  task  and  the  questionnaire,  and  informal  dis- 
cussions of  warranty  impact,  barriers,  and  contractor  experiences. 
At  these  meetings,  the  questionnaire  was  presented  to  the  con- 
tractor program  manager  for  later  retrieval.  Any  additional 
questions  concerning  the  format  or  data  requirements  were  also 
addressed.  A number  of  secondary  visits  were  also  made  to  se- 
lected contractors  during  Phase  II  to  address  additional  issues, 
retrieve  questionnaires  that  had  not  yet  been  returned,  or  clar- 
ify answers  to  specific  questions.  (Appendix  E contains  the 
Government  and  contractor  interview  schedule. ) A total  of  six- 
teen of  the  seventeen  questionnaires  distributed  during  the  in- 
vestigation were  completed  and  returned  for  analysis. 

Phase  III,  data  analysis  and  report  preparation,  began  16 
June  and  was  concluded  with  delivery  of  the  draft  final  report 
on  1 October  1976.  The  analysis  consisted  of  combining  the  re- 
sults of  the  many  interviews  with. the  formal  questionnaire 
responses  in  an  effort  to  deduce  the  collective  warranty  impact. 

In  order  to  preserve  the  proprietary  and  sensitive  nature  of 
many  of  the  contractor  interviews  and  questionnaires,  all  specif- 
ic references  to  contractors  have  been  eliminated  throughout 
the  report.  When  contractor  data  taken  from  questionnaires 
are  displayed,  coded  alphanumeric  identifiers  are  used. 

E.  REPORT  ORGANIZATION 

This  report  has  been  organized  to  follow  the  logic  and 
format  established  for  the  study  tasks  as  reflected  by  the 
contractor  questionnaire. 
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Chapter  II  identifies  the  principal  features  of  the  war- 
ranty contracts  and  addresses  those  elements  of  the  requirements 
that  directly  bear  upon  either  the  design  of  the  equipment  or 
the  conduct  of  the  development  program.  The  chapter  also  sum- 
marizes the  specific  contract  features  and  warranty  requirements 
for  the  candidate  programs. 

Chapter  III  presents  background  information  relating  RIW 
contract  requirements  to  specific  programs  and  contractors. 

This  chapter  also  describes  the  environment  and  conditions  under 
which  the  warranty  contract  obligations  were  established. 

Chapters  IV,  V,  VI  present  the  data,  analyses,  and  findings 
resulting  from  the  many  interviews  and  the  questionnaire 
responses.  Chapter  IV  examines  the  actual  reported  impact  of  the 
warranty  requirements  on  the  candidate  development  programs. 
Program  impacts  are  described  and  coupled  with  the  driving  re- 
quirements wherever  possible;  barriers  to  the  achievement  of 
greater  impacts  are  also  identified.  Chapter  V extends  the 
findings  of  the  previous  chapter  to  potential  development  pro- 
gram impacts  under  selected  conditions  relating  to  warranty  ap- 
plication decisionmaking  and  contractor  specification  flexi- 
bility. Chapter  VI  addresses  the  identifiable  design  changes 
in  the  candidate  equipment  resulting  from  warranty  considera- 
tions. These  design  changes  are  described  in  both  specific 
and  generic  terms  as  the  data  retrieved  fFJqm  the  contractors 
permit . 

Chapter  VII  addresses  a number  of  important  additional 
considerations  relevant  to  warranty  applications  and  consequent 
impacts  that  were  identified  during  the  program  office  and  con- 
tractor interviews.  Some  of  these  issues  are  believed  to  be 
capable  of  masking  potential  impacts  that  were  identified  in 
the  preceding  chapters. 

Chapter  VIII  summarizes  the  findings  of-  the  research  effort 
and  presents  the  conclusions  derived  from  those  findings. 
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Guidelines  for  continued  warranty  policy  development  are  pre- 
sented for  OSD  and  Service  consideration  in  Chapter  IX.  Finally, 
Chapter  X identifies  a number  of  areas  in  which  continued  study 
would  lead  to  a greater  understanding  of  the  warranty  contract 
environment  and  how  it  relates  to  the  development  planning 
process . 


II 


EQUIPMENT  WARRANTY  REQUIREMENTS 

Section  A of  this  chapter  describes  the  general  features 
of  reliability  improvement  warranties  and  companion  guarantees.1 
Section  B summarizes  the  specific  warranty  terms  and  conditions 
for  each  of  the  candidate  subsystems  as  a general  frame  of 
reference  for  the  subsequent  analysis. 

It  should  be  noted  that  specific  requirements  for  warran- 
ties vary  according  to  their  application.  And  like  the  candi- 
date applications,  many  of  the  warranty  provisions  are  truly 
experimental  in  nature.  Lessons  learned  during  the  initial 
trial  procurements  could  alter  future  warranty  requirements  and 
could  also  lead  to  negotiated  changes  in  existing  contract 
arrangements . 

Investigations  of  the  candidate  subsystems  in  this  study 
revealed  that  frequently  the  basic  warranty  requirement  was 
accompanied  by  other  guarantees  either  as  an  Integral  part  of 
the  contract  or  as  options  to  be  exercised  by  the  Government. 
Basically,  these  guarantees  fell  into  two  major  categories, 
guarantees  for  operational  mean  time  between  equipment  failure 
(MTBF),  and  guarantees  for  Service  logistics  support  costs.2 
Because  most  of  the  candidate  subsystems  were  subject  to  the 
possibility  of  more  than  one  type  of  warranty/guarantee  during 
their  development  program,  each  type  will  described. 

^ore  detailed  discussions  can  be  found  in  several  excellent  and  comprehen- 
sive reports  prepared  by  ARINC  Research  Corporation.  A recent  sunmary 
report  is  especially  recommended : Guidelines  for  Application  of  Warranties 
to  Air  Force  Electronic  Systems , ARINC  Pub.  No.  1500-01-1-1^51  (Annapolis, 
Md.,  1975). 

2The  F-l6  avionics  contracts  contain  firm  requirements  or  options  for  each 
of  these  types  of  guarantees. 
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A.  GENERAL  CONTRACT  REQUIREMENTS 
1 . Reliability  Improvement  Warranty 

A reliability  improvement  warranty  is  basically  a contrac- 
tual agreement  whereby,  for  a fixed  price,  the  contractor  agrees 
to  repair  all  failed  units  (except  for  explicit  exclusions — 
see  below)  returned  to  him  during  a specified  period  of  time. 

The  terminology  relating  to  reliability  Improvement  stems  from 
the  incentives  for  improving  equipment  reliability  available  to 
the  contractor  both  during  development  and  after  field  deploy- 
ment. Some  of  the  key  requirements  and  features  of  the  RIW 
are  described  in  the  following  paragraphs. 

a . Warranty  Period 

The  warranty  period  is  the  time  span  during  which  the  war- 
ranty is  in  effect.  This  period  can  be  measured  in  total  calen- 
dar time,  beginning  at  equipment  delivery;  a fixed  calendar 
time  period,  independent  of  delivery  schedules  and  production 
rates;  or  equipment  operating  time,  measured  either  by  indi- 
vidual equipment  operating-time  indicators  or  on  an  aggregate 
population-usage  basis.  Most  warranty  periods  cover  a time 
period  of  three  or  more  years,  during  which  some  of  the  older 
equipment  is  expected  to  require  normal  maintenance  service 
and  repair. 

b.  Excluded  Failures 

Most  warranty  requirements  contain  provisions  for  excluded 
failures.  Equipment  failures  that  can  be  traced  to  misuse  or 
mistreatment,  for  example,  are  usually  excluded  from  warranty 
coverage.  A number  of  reasons  for  possible  exclusions  are 
listed  below. 

• Fire 

• Explosion 

• Submersion 


• Flood  and  other  acts  of  God 

• Aircraft  (vehicle)  crash 

• Enemy  action 

• Seal  broken  on  unit  while  outside  contractor's  control 

• External  physical  damage  caused  by  accidental  or  wilfull 
mistreatment 

• Internal  physical  damage  caused  by  accompanying  external 
physical  damage  due  to  mistreatment  or  to  tampering  by 
noncontractor  personnel 

• Induced  failures,  i.e.,  failures  of  hardware  items  in- 
duced by  malfunction  or  improper  operation  of  outside 
(system  interfacing)  units 

• Consequential-incidental  damages 

• Improper  installation,  operation,  or  maintenance. 

c .  Unverified  Failures 

Reliability  warranties  have  provisions  to  accommodate  those 
units  returned  under  warranty  that  subsequently  are  found  to 
operate  properly  (RTOKs).  The  parameters  of  this  requirement 
relate  to  the  percentage  of  returns  that  test  good,  the  incurred 
costs  that  are  included  in  the  warranty  price,  and  the  costs  to 
be  paid  per  unverified  failure  above  the  percentages  covered  by 
the  warranty  agreement.1 


d . Usage  Rate  Adjustment 

Most  warranties  are  negotiated  and  priced  on  the  basis  of 

an  expected  usage  rate  over  the  warranty  period.  Significant 

deviations  from  the  intended  usage  rate  require  negotiated 

warranty  price  adjustments  to  account  for  the  change  in  expected 

failure  rates  and  maintenance  costs. 

/ 

e . Engineering  Change  Proposals 

One  of  the  primary  incentives  of  a warranty  is  contractor 
freedom  to  make  continuous  reliability  improvements  in  the 
design  of  the  equipment.  The  provision  that  the  Government  will 

1Typically,  contractors  are  asked  to  price  RTOK  return  rates  14)  to  25  or 
30  percent  of  total  returns.  The  Government  usually  negotiates  a nominal 
test  and  reissue  price  ($100/unit)  for  additional  units  that  test  good  at 
the  contractor's  plant. 


accept  no-cost  engineering  change  proposals  (ECPs)  that  do  not 
alter  equipment  form,  fit,  or  function  permits  contractor 
design- improvement  latitude  with  the  only  restriction  that  all 
equipment  be  modified  to  the  latest  configuration  by  the  end  of 
the  warranty  period. 

f . Turn-around  Time 

An  important  RIW  requirement,  turn-around  time  (TAT), 
specifies  the  maximum  time  allowable  to  repair  (and  place  in 
storage)  failed  units  at  the  contractor's  plant  (or  to  ship  a 
replacement  spare).  Since  field  availability  is  affected  by 
delays  in  repair  times,  penalties  are  normally  assigned  through 
liquidated  damage  payments  or  consignment  spares  in  order  to 
keep  equipment  turn-around  time  within  set  time  intervals. 

2.  Mean  Time  Between  Failure  Guarantees 

One  type  of  guarantee  that  has  accompanied  RIWs  and,  when 
included,  must  be  considered  a part  of  any  contractor  response 
stimuli,  is  the  guaranteed  MTBF.  Under  this  type  of  warranty, 
the  contractor  stipulates  that  the  delivered  equipment  will 
attain  or  exceed  predetermined  values  for  field  reliability. 

The  usual  penalty  for  failure  to  achieve  the  predicted  relia- 
bility has  been  contractor  provision  of  consignment  spares  to 
replace  failed  units,  thereby  maintaining  field  availability. 
For  most  of  the  candidate  programs,  MTBF  guarantees  have  been 
incorporated  as  an  additional  option,  subsidiary  to  the  basic 
RIW  option. 1 

The  basic  requirements  for  the  MTBF  guarantee  consist  of 
specified  minimum  performance  levels  as  a function  of  equipment 
reliability  maturation,  the  method  of  determining  or  measuring 
achieved  MTBFs,  and  the  term  of  the  guarantee,  usually  set  at 
time  periods  marked  by  the  achievement  of  specified  MTBF  levels. 

iAIl  the  F-16  programs  have  guaranteed  MTBF  options  as  an  additional  option 
to  be  exercised  beyond  the  RIW. 
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The  principal  contractor  risk  element  of  an  MTBF  guarantee 
is  the  cost  "penalty”  associated  with  providing  consignment  spares 
if  specified  MTBF  levels  are  not  achieved  and  maintained. 

3.  Logistics  Support  Cost  Guarantee 

The  third  type  of  equipment  guarantee  or  warranty  is  the 
logistics  support  cost  commitment,  wherein  the  estimated 
Government  logistics  support  cost  is  computed  by  means  of  a 
standard  negotiated  cost  model  using  measured  field  cost, 
reliability,  and  maintainability  values.  The  difference  between 
the  contractual  target  logistics  support  cost  (TLSC)  and  the 
measured  logistics  support  cost  (MLSC)  determines  the  basis  for 
contractor  incentive  fees  or  "penalties,"  which  normally  include 
provisions  for  the  "correction  of  deficiencies ." 1 

The  warranty  or  guarantee  aspect  of  the  logistics  support 
cost  commitment  lies  in  contract  "penalties"  for  failure  to 
achieve  a threshold  MLSC.  In  some  applications,  the  Government 
can  demand  additional  equipment  in  lieu  of  penalty  payments, 
which  is  similar  to  the  requirement  for  contingency  spares  under 
guaranteed  MTBF  programs.  Additional  aspects  of  the  LSC  guarantee 
that  are  similar  to  RIWs  and  guaranteed  MTBF  are  provisions  for 
correction  of  deficiencies  if  the  equipment  does  not  meet  certain 
performance  thresholds,  such  as  MTBF. 

The  LSC  concept  is  another  means  for  the  Government  to 
obtain  contractual  guarantees  that  the  delivered  equipment  will 
meet  specified  levels  of  reliability  and  maintainability.2  The 


lrrhe  correction  of  deficiencies  (COD)  clause  takes  on  greater  significance 
here  since  it  becomes  the  forcing  clause  to  achieve  specified  reliability 
goals,  thereby  achieving  a minimum  MISC. 

2lhe  LSC  formulas  usually  are  dominated  by  the  three  major  variables:  acqui- 

sition cost;  field  reliability,  e.g.,  MIBF;  and  field  maintainability, 
e.g. , mean  time  to  repair  (MTTR).  The  relative  importance  of  these  vari- 
ables depends  upon  the  equipment  unit  cost,  the  number  of  units  produced, 
the  level  of  field  reliability,  and  the  complexity  of  repair  actions. 
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obligation  of  the  contractors  is  limited  over  time,  provided  that 
the  sampling  plan  for  measuring  LSC  yields  favorable  results; 
however,  if  the  COD  provisions  are  activated  and  the  desired  LSC 
is  not  achieved  by  contractor  actions,  the  time  period  of  con- 
tractor guarantee  obligation  in  many  cases  could  be  extended 
throughout  the  equipment  lifetime. 

B.  CANDIDATE  PROGRAM  RIW  REQUIREMENTS 

This  section  summarizes  the  warranty  requirements  that  were 
either  imposed  upon  the  contractors  of  the  candidate  subsystems 
as  firm  requirements  or  as  options  available  for  exercise  by  the 
Government.  These  requirements  are  presented  in  Table  1 in  three 
major  groupings:  RIW,  guaranteed  MTBF,  and  logistics  support 

cost  gaurantees. 

It  should  be  noted  that  eight  of  the  eleven  candidate  pro- 
grams are  still  in  the  engineering  development  stage  and  changes 
in  the  requirements  are  still  possible  prior  to  contractual  com- 
mitment.1 It  should  also  be  noted  that  the  Navy  design-to-price 
EW  suite  program  did  not  include  specifically  defined  RIW 
requirements,  but  both  contractors  (Hughes  and  Raytheon),  were 
notified  by  the  program  office  that  RIW  requirements  were  under 
consideration  for  selected  portions  of  the  system  and  that  they 
should  analyze  their  designs  for  accommodation  to  RIW  condi- 
tions. The  latest  version  of  the  production  Request  for 
Proposal  (RFP ) available  to  the  investigators  requested  that 
the  contractors  propose  RIW  as  an  alternate  maintenance  concept 
if  they  could  demonstrate  life-cycle  cost  savings.  Thus, 
despite  the  fact  that  specific  RIW  requirements  had  not  been 
defined  for  the  competing  systems,  both  contractors  were 
designing  and  developing  equipment  with  the  knowledge  that  RIW 
requirements  were  a potential  production  contract  ingredient. 

lrThe  requirements  shown  in  Table  1 were  current  as  of  1 June  1976.  They  are 
believed  to  be  accurate  indicators  of  the  final  requirements  that  contrac- 
tors were  anticipating  for  production  award  comnitment. 


Proprietary  data  available  to  IDA  i dicate  that  considerable 
analytical  effort  was  expended  by  t e contractors  to  determine 
potential  RIW  applicability,  subcon  ract  influence,  financial 
risk,  logistics  considerations,  and  ultimate  support-cost  rela- 
tionships vis-a-vis  organic  mainten  nee.  Therefore,  while  the 
RIW  requirements  cannot  be  explicit  y delineated  for  this 
system,  the  work  of  the  contractors  convinced  the  investigators 
that  this  system  should  be  retained  as  one  of  the  candidates 
for  analysis. 

1 . RIW  Requirements 

As  illustrated  in  Table  1,  nine  of  the  eleven  candidate 
programs  contain  explicitly  defined  requirements  for  RIW  ob- 
ligations during  the  engineering  development  program.  For 
those  nine  programs,  the  RIW  is  an  option  to  be  exercised, 
based  upon  the  contractor  bid  at  the  time  of  production  contract 
award. 

Most  of  the  RIW  requirements  ,ire  similar  for  the  nine 
candidate  programs  involved.1  All  but  the  B-l  ECM  suite  have 
warranty  periods  of  four  years  or  longer,  have  provisions  for 
no-cost  ECPs,  and  have  specified  equipment  turn-around  times 
with  penalties  for  late  delivery. 

The  principal  differences  in  the  RIW  requirements  are  in 
approval  time  for  no-cost  ECPs  and  the  coverage  of  unverified 
failures.  The  subcontractors  for  the  F-16  subsystems  must 
wait  65  days  for  approval  before  they  can  unilaterally  in- 
corporate no-cost  ECPs,  whereas  the  Air  Force  Tacan  and  the 
Army  altimeter  and  doppler  navigation  system  contractors  must 
wait  30-35  days.  This  difference  is  the  result  primarily  of  the 
subcontractor  relationship  between  the  F-l6  equipment  contractors 


lrrhis  is  not  surprising,  since  ARINC  Research  Corporation,  working  as  a 
Service  consultant,  was  the  architect  of  most  contractual  clauses. 
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and  the  prime  contractor,  General  Dynamics.  The  latter's  con- 
tract calls  for  ECP  incorporation  30  days  after  receipt  by  the 
Government  unless  written  notification  of  non-approval  is  given. 

Unverified  failures  are . completely  covered  by  the  RIW  re- 
quirements for  all  but  two  candidates — the  Air  Force  Tacan  and 
the  Army  doppler.  For  these  two  systems,  a coverage  limit  has 
been  established  (30  and  25  percent,  respectively)  and  addi- 
tional unverified  failures  are  sources  of  nominal  reimbursement 
to  the  contractors.1 

In  general,  the  RIW  requirements  of  the  candidate  programs 
were  characterized  by  their  similarities.  They  represented  long 
term  obligations  on  the  part  of  contractors  to  repair  and  main- 
tain equipment  returned  from  the  field  and  to  guarantee  equip- 
ment availability  through  specified  turn-around  times  and  "penal 
ties"  for  delay.  In  every  case,  these  requirements  remained  as 
priced  options  that  the  Government  had  the  freedom  to  exercise 
at  the  time  of  production  contract  award. 

2 . Guaranteed  MTBF  Requirements 

Seven  of  the  candidate  subsystems,  including  all  five  of 
the  subcontracted  F-16  subsystems,  are  being  developed  with  con- 
tractual options  for  guaranteed  MTBF  provisions.  Each  of  the 
seven  subsystems  is  potentially  under  the  same  general  require- 
ments, which  include  total  unit  sampling,  consignment  spares 
supply,  and  minimum  demonstration  time  periods.  Specific  MTBF 
values,  as  guaranteed,  are  shown  in  Table  2. 

The  guaranteed  MTBF  provisions  for  the  F-16  equipment  are 
slightly  different  from  the  provisions  for  the  Tacan  and  doppler 


Unverified  failures  are  a serious  concern  of  contractors  because  past 
experience  in  avionics  has  shown  that  20-40  percent  of  all  returned  units 
test  good  at  the  contractor's  plant.  The  nominal  fee  paid  for  each  unit 
returned  over  the  threshold  does  not  usually  cover  test  and  turn-around 
costs. 
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Table  2.  MTBF  GUARANTEES  FOR  CANDIDATE  SUBSYSTEMS 


Candidate  Equipment 

Unit 

MTBF  Guarantee 
(Hours ) 

1st 

Year 

2nd 

Year 

3rd 

Year 

F-16 

Flight  Control  Computer 

162 

242 

260 

F-16 

Radar  E/0  Display 

155 

228 

244 

F-16 

Fire  Control  Computer 

415 

600 

640 

F-16 

Fire  Control  Radar* 

70 

82 

90 

F-16 

Navigation  System 

185 

284 

300 

USAF 

ARN-118  Tacan 

500 

625 

800 

USA 

ASN-128  Doppler 

500 

500 

500 

*Radar  guarantees  are  based  on  individual  LRU  MTBF  values. 


navigation  subsystems  in  that  the  guarantee  is  an  option  to  be 
exercised  only  after  the  decision  is  made  to  select  the  RIW;  the 
Government  may  select  the  RIW  with  or  without  the  guaranteed 
MTBF,  but  the  guaranteed  MTBF  option  cannot  be  exercised  in- 
dependently of  the  RIW.  In  the  Tacan  and  doppler  navigation 
contract  clauses,  the  guaranteed  MTBF  is  an  integral  part  of 
the  total  warranty  requirements  and  is  exercised  automatically 
if  the  RIW  option  is  selected. 

3.  Logistics  Support  Cost  Commitment 

A logistics  support  cost  commitment  is  a potential  contrac- 
tual requirement  for  all  the  F-16  subsystem  candidates  and  was 
an  ingredient  of  the  ARC-164  radio  and  ARN-118  Tacan  development 
contracts.1  These  requirements  are  similar  in  nature  to  the 
preset  target  logistics  support  cost  values  derived  through  the 
Air  Force  logistics  command  (AFLC)  cost  formula.2  Field 


lrThe  LSC  commitment  for  the  F-16  equipment  is  a firm,  basic  contract  require- 
ment unless  the  RIW  option  is  selected. 

2 See  G.  Harrison,  The  Development  and  Analysis  of  RIW  and  COD  Provisions  for 
the  Air  Combat  Fighter  (ACF)  Aircraft , Pub.  No.  (continued  on  next  page) 
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measurements  demonstrating  the  achievement  of  the  LSC  will 
determine  contractor  profit  or  loss.1 

4.  Candidate  Requirements  Summary 

An  overview  of  the  warranty  requirements  for  the  candidate 
programs  shows  that  all  of  the  programs  are  being  developed  with 
either  explicitly  defined  or  implicitly  understood  requirements 
for  contractor  guarantees  for  equipment  field  reliability.  Nine 
of  the  eleven  programs  include  specific  options  for  an  RIW,  and 
seven  of  those  nine  also  include  provisions  for  MTBF  guarantees. 
Seven  programs  contain  potential  requirements  for  logistics 
support  cost  commitments  or  guarantees. 

Most  of  the  guarantee  requirements  are  similar  among  the 
candidate  subsystems.  The  earlier  ARC-164  radio  program  was  the 
prototype  for  application  of  logistics  support  cost  guarantees 
and  the  ARN-118  Tacan  program  was  the  prototype  for  the  applica- 
tion of  long-term  RIWs  and  guaranteed  MTBF  provisions.  Subse- 
quent programs  adopted  the  prototype  contract  formats.  All  of 
these  requirements  (explicit  and  implicit)  focus  attention  on 
the  Government's  desire  to  minimize  operational  support  costs 
and  maximize  equipment  availability  through  increased  equipment 
reliability  and  maintainability. 


(continued)  1264-01-11370  (Annapolis,  Md. : ARINC  Research  Corp. , 1975)  for 
more  information  about  the  AFLC  cost  model  and  the  F-l6  ISC  provisions. 

lr[he  ISC  corrmitment  is  really  a guarantee  of  field  reliability  and  maintain- 
ability, to  the  extent  that  these  items  drive  the  ISC  value.  The  major 
differences  between  this  corrmitment  and  the  RTW-MTBF  provisions  are  the 
risks  associated  with  organic  maintenance,  the  validity  of  the  field 
measurements,  and  the  protection  afforded  by  the  allowable  interval  asso- 
ciated with  the  target  logistics  support  cost  values. 
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III 

THE  WARRANTY  INTRODUCTION  PROCESS  AND 
CONTRACTOR  ENVIRONMENT 


Previous  IDA  research  into  electroriics  subsystem  acquisi- 
tion policy  and  preliminary  discussions  with  both  Government 
personnel  and  industrial  contractors  indicate  that  the  impact 
of  the  new  warranty  policies  is  influenced  to  a great  extent  by 
the  characteristics  of  policy  application.1  The  earlier  the 
policies  are  applied  in  an  equipment  development  program,  the 
longer  time  the  policies  will  have  to  influence  the  equipment 
design  or  development  program  planning.  The  contractor  response 
to  the  policies  for  warranty  application  m^y  also  be  affected 
by  many  factors,  including  (a)  the  manner  in  which  the  warranty 
requirements  are  formally  applied  to  the  program  and  (b)  the 
contractor's  past  exposure  to  equipment  warranties  or  operational 
performance  guarantees. 

A.  INTRODUCTION  OF  WARRANTY  REQUIREMENTS 


The  introduction  of  warranty  requirements  was  found  to  be 
a two-step  process  consisting  of,  first,  informal  discussions 
between  the  Government  program  officers  and  the  equipment  con- 
tractors and,  second,  formal  introduction  of  the  potential 
warranty  requirements  as  contract  elements.  The  formal  state- 
ments of  future  contract  requirements  usually  surface  as  draft  , 
statements  that  eventually  are  refined  and  incorporated  into  a 
formal  Request  for  Quotation  (RFQ)  or  Request  for  Proposal  (RFP). 


iThis  was  shown  to  be  a key  variable  of  the  design-to-cost  acquisition 
policies  investigated  previously  by  IDA  and  reported  In  C.  David  Weimer, 
The  Application  of  Design- to- Cost  Acquisition  Policies  to  Selected  Elec- 
tronics Subsystem  Development  Programs , IDA  Study  S-459  (1975) • 
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Since  contractors  are  able  to  -espond  most  positively  to 
formal  statements  of  new  contract  r -quirements,  the  IDA  research 
questionnaire  was  designed  to  ident  .fy  the  point  in  the  develop- 
ment programs  at  which  the  warranty  requirements  were  formalized. 
Such  identification  was  believed  to  present  a consistent  measure- 
ment of  an  event  that  would  precipi  ;ate  a contractor  response, 
which  in  turn  would  reflect  a desigi  or  development  program 
impact . 

Of  the  eleven  candidate  progra ns  examined  during  this  study, 
nine  contain  explicit  mention  of  future  equipment  warranty  possi- 
bilities in  their  engineering  development  program  RFP.  In  some 
instances,  such  as  in  the  basic  F-lo  full-scale  development  RFP, 
warranty  provisions  were  added  as  a requirement  only  in  the  final 
versions.  Several  draft  RFPs  were  prepared  and  transmitted  to 
the  prospective  prime  contractors  and  their  subcontractors  with- 
out any  mention  of  RIW  possibilities.1  Inclusion  of  RIW  and 
guaranteed  MTBF  requirements  came  at  a point  when  the  contractors 
had  little  time  to  react  and  consider  fully  the  future  implica- 
tions for  the  equipment  design,  the  development  program  plan, 
and  cost. 

While  the  introduction  of  warranty  requirements  occurred 
most  frequently  as  early  as  the  development  program  RFP,  the 
final  negotiation  of  the  specific  contract  requirements  did  not 
usually  occur  until  some  time  later.  The  experience  of  the 
sixteen  questionnaire  respondents  is  illustrated  in  Figure  3. 

Six  contractors  negotiated  the  terms  and  conditions  of  the 
warranty  contract  provisions  during  the  negotiations  for  the 
combined  engineering  development  and  production  contracts.2 

JIt  is  believed  that  the  concept  of  guarantees  for  logistic  support  costs  was 
part  of  the  early  or  initial  program  planning.  The  RIW  and  guaranteed  MIBF 
requirements  were  added  during  the  final  revisions  to  the  RFP. 

2These  engineering  development  contracts  contained  fixed-price  options  for 
production  quantities  that  essentially  specified  the  terns  and  conditions 
for  production.  There  is  not  to  be  any  separate  production  RFP  award. 
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Figure  3.  WARRANTY  REQUIREMENT  APPLICATIONS  SUMMARY 


Seven  contractors  were  formally  notified  of  a potential  warranty 
requirement  during  the  development  RFP  but  were  not  required 
to  negotiate  detailed  requirements  until  the  production  contract. 
The  remaining  three  contractors  did  not  formally  learn  of  war- 
ranty requirements  until  the  production  RFP.1  The  contractor 
survey  shows  that  most  (thirteen  out  of  sixteen)  learned  of  the 
Government's  intention  to  consider  an  equipment  warranty  as  a 
result  of  the  development  RFP.  However,  seven  of  those  contrac- 
tors did  not  know  what  the  warranty  requirements  would  ultimately 
be.  And  each  of  these  thirteen  contractors  was  developing  equip- 
ment with  option  possibilities  for  warranty  exercise  rather  than 
firm  requirements  for  warranty  incorporation. 

B.  CONTRACTOR  DEVELOPMENT  PROGRAM  STATUS 

The  development  program  status  of  each  candidate  contractor 
was  also  determined  by  the  questionnaire  in  an  effort  to  qualify, 
by  present  program  experience,  the  responses  describing  program 
impact.  It  was  found  that  seven  of  the  contractors  had  completed 
their  product  development  programs  at  the  time  of  their  response 
(1  June  1976).  The  other  contractors'  development  programs  were 
from  10-85  percent  complete.  The  average  program  status  of 
those  still  in  the  development  stage  at  the  time  of  their  ques- 
tionnaire response  was  approximately  50  percent  complete. 

A correlation  of  program  status  and  application  of  warranty 
requirements  is  presented  in  Figure  4.  The  six  contractors 
who  negotiated  warranty  requirements  prior  to  development  con- 
tract award  could  assess  and  report  the  warranty  impact.  Simi- 
larly, the  seven  contractors  who  finalized  the  warranty  require- 
ments during  development  could  report  the  impact  of  the  require- 
ments as  they  were  formulated  during  the  development  phase.  The 


1 These  results  relate  to  formally  transmitted  warranty  requirements  or  state- 
ments of  work  for  potential  pricing  and  bidding. 
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remaining  three  contractors  were  obligated  to  report  the  impact 
that  might  have  occurred  had  they  known  prior  to  the  production 
RFP  that  warranty  provisions  were  to  be  priced  as  an  optional 
contractual  requirement. 


Timing  of  Formal 
Warranty  Requirements 
Application 

Number 

of 

Contractors 

Number  of 
Contractors 
With  Completed 
Development  Programs 

Number  of 
Contractors 
Still  In 
Development 

Before  Development 
Contract  Award 

6 

2 

4 

During  the 

Development 

Program 

7 

4 

3 

At  Production 
Contract  Award 

3 

1 

2* 

*These  two  contractors  were  the  only  respondents  who  did  not  have 
opportunity  to  experience  and  report  their  actual  program  warranty  impact. 

Figure  4.  APPLICATION  OF  WARRANTY  REQUIREMENTS 
AND  CURRENT  PROGRAM  STATUS 


C.  EQUIPMENT  TECHNOLOGICAL  STATUS 

The  design  and  execution  of  product  development  programs 
depend  in  large  measure  on  the  technical  difficulties  to  be 
surmounted.  Whether  the  product  represents  an  improvement  over 
predecessor  equipment  or  is  a new  technological  development  can 
also  determine  the  degree  of  development  risk  to  be  managed 
during  the  program,  as  well  as  the  ability  of  the  contractor  to 
predict  future  field  reliability  and  maintainability  from 
development  program  test  results.  Thus,  the  impact  exerted  by 
future  requirements  for  product  warranties  could  easily  be  en- 
hanced or  attenuated,  depending  on  the  technical  risks  and 
product  performance  uncertainties  to  be  addressed  during 
development . 
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The  technical  status  of  contractor  equipment,  as  determined 
by  the  questionnaire  is  shown  in  Figure  5.  A majority  of  the 
respondents  characterized  their  product  as  a new  technological 
development  rather  than  a product  improvement.1  Other  factors 
being  equal,  it  is  reasonable  to  expect  that  these  development 
programs  contained  a significant  amount  of  technical  risk,  and 
by  their  very  nature,  little  previous  product  reliability  or 
maintainability  data  would  be  directly  relevant  to  efforts  to 
predict  equipment  operational  performance  with  confidence. 


Figure  5.  SUMMARY  OF  CONTRACTOR  ASSESSMENT  OF  EQUIPMENT 
DEVELOPMENT  CLASSIFICATION 


^is  response  was  unexpected  because  many  of  the  equipment  developments 
that  were  characterized  by  the  Government  program  offices  as  product 
improvements  were  not  classified  as  such  by  the  contractor. 
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CONTRACTOR  WARRANTY  EXPERIENCE 
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Another  factor  that  could  be  critical  in  the  warranty  appli- 
cation and  response  is  the  contractor's  previous  experience  with 
equipment  warranties.1  We  expected  that  if  the  contractors  had 
previous  experience  with  the  negotiation  and  execution  of 
multi-year  warranties  many  of  the  warranty  impacts  would  be 
easier  to  identify  during  succeeding  programs. 

Responses  to  our  inquiry  concerning  previous  warranty 
experience  indicated  that  only  three  of  the  sixteen  respondents 
had  acquired  multi-year  warranty  experience  on  predecessor 
equipment.  Of  the  three,  only  one  contractor  had  relevant 
military  experience.  The  other  two  contractors  were  producing 
similar  equipment  for  both  the  military  and  civilian  markets 
and  had  experience  with  the  airline  warranty  environment  on 
the  civilian  side  of  their  business.  However,  this  experience 
was  not  readily  available  or  transferable  to  the  candidate  pro- 
grams because  of  organizational  barriers  or  differences  in 
equipment  operating  environment. 

These  findings  reveal  that  the  warranty  proposals  and  the 
eventual  contracts  were,  except  for  a small  minority,  the  first 
contractual  exposure  most  of  the  contractors  had  to  multi-year 
warranties.  Generally,  there  was  little  internal  corporate 
experience  with  the  warranty  concept  and  thus  very  little  basis 
for  adopting  a design  approach  or  for  structuring  a development 
program  using  past  warranty  experience. 

E.  WARRANTY  CONTRACT  NEGOTIATION  DIFFICULTIES 

The  research  thus  far  has  shown  that  negotiation  of  war- 
ranty contract  provisions  occurred  throughout  the  typical 


!We  were  only  interested  in  the  special  case  of  multi-year,  failure- free 
repair  or  maintenance  warranties.  One-year  warranties  for  correction  of 
deficiencies  or  defects  were  not  considered  part  of  relevant  experience. 
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development  program  and.  In  many  cases,  was  not  finalized  until 
the  production  contract.  It  was  also  found  that  the  candidate 
programs  were  predominately  characterized  by  the  contractors 
as  being  new  technological  developments,  rather  than  product 
improvements.  And  finally,  the  introduction  of  multi-year 
warranty  provisions  was  a new  experience  for  most  of  the 
candidate  contractors.  This  environment  must  now  be  inte- 
grated with  the  negotiation  experiences  reported  by  the  candidate 
contractors  who  had  successfully  negotiated  warranty  contracts, 
or  who  were  currently  in  the  negotiation  process. 

Based  upon  our  initial  contractor  interviews,  a decision 
was  made  to  Inquire  as  to  which  contract  areas  were  the  most 
critical  during  the  warranty  negotiation.  This  approach  was 
adopted  in  the  belief  that  identification  of  critical  negotiat- 
ing areas  would  be  positive  Indicators  of  changes  in  equipment 
design  or  development  planning.  Especially  when  warranty 
negotiations  occurred  early  in  the  development  program,  it  was 
thought  that  innovations  or  changes  in  design  and  development 
activities  should  be  observable  that  would  provide  greater 
contractor  confidence  about  achieving  the  most  difficult  con- 
tractual commitments.  Ten  typical  problem  areas,  derived  from 
the  Initial  contractor  discussions,  were  presented  in  the 
questionnaire  and  space  was  provided  for  additional  entries. 

Contractor  responses  to  this  part  of  the  questionnaire 
are  shown  in  Table  3-  As  Indicated  by  the  table,  twelve  of  the 
contractors  reported  that  the  measurements  of  field  reliability 
was  a critical  area.  Closely  related  to  this  area  is  "defini- 
tion of  relevant  failures,"  which  ranked  second  with  ten 
responses.  Areas  of  "warranty  price"  and  equipment  "turn- 
around time"  also  were  critical  areas  for  at  least  half  of 
the  respondents. 

Concern  about  the  measurement  of  actual  field  reliability 
is  related  to  identifying  failures  that  can  be  considered 
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relevant.  Because  of  these  findings,  the  design  of  the  equip- 
ment can  be  expected  to  reflect  efforts  to  identify  or  verify 
relevant  failures  quickly.  Also,  the  design  should  reflect  a 
concern  for  accurately  measuring  equipment  operating  times. 
Additional  development  test  time  to  verify  these  design  features 
might  also  be  expected. 

Turn-around  time  was  of  great  concern  to  the  contractors 
because  of  the  penalties  that  could  be  imposed  if  negotiated 
times  were  exceeded.  Batch  shipments  of  failed  units,  for 
example,  could  disrupt  the  normal  production  process,  and 
costly  replacement  or  provision  of  contingency  spares  could 
also  be  required  to  maintain  a smooth  pipeline  flow.  Our  analy- 
sis of  turn-around  time  indicates  a need  for  more  contractor 
study  of  the  proposed  equipment  inspection  and  repair  process, 
as  well  as  the  sources  and  financial  impact  of  potential  delays 
in  turn-around  time. 

Warranty  price  was  a problem  area  because  price  level 
determines  the  reliability  break-even  point  for  achieving  the 
incentive  profit  as  well  as  the  maximum  and  total  incentive 
fee  available  to  the  contractor  over  the  warranty  term.1  The 
proper  balance  of  risk  and  reward  is  therefore  of  major  concern 
to  all  contractors. 2 

F.  FINDINGS 

This  chapter  has  reported  research  findings  in  the  areas 
of  warranty  requirements  application,  subsystem  technological 


lrThis  is  not  to  imply  an  incentive-fee  type  warranty  contract.  Incentive 
profit  can  be  achieved  by  reducing  fixed-price  warranty  costs  below 
estimated  costs. 

2It  is  the  opinion  of  the  investigators  that  warranty  price  negotiations 
have  presented  difficulties  primarily  because  the  Government's  calculation 
of  the  warranty  "should  cost"  value  does  not  discount  guaranteed  equipment 
reliability  values  for  field  operations  and  does  not  permit  profit  or  fee 
levels  high  enough  to  compensate  for  risk. 


status,  contractor  warranty  experience,  and  critical  areas  of 
warranty  negotiation.  These  variables,  associated  with  the 
application  of  the  warranties  and  the  conditions  within  the 
contractor  environment,  are  believed  to  be  important  considera- 
tions when  the  impact  of  warranties  upon  equipment  design  and 
development  program  execution  is  investigated. 


Our  research  indicates  that  while  most  of  the  contractors 
were  made  aware  of  the  Government's  intention  to  consider 
warranties  as  a production  contract  option,  the  terms  and  condi- 
tions of  the  warranty  were  frequently  not  finalized  until  the 
production  contract  negotiations.  of  the  subsystem  contractors 
examined  in  this  study,  all  but  two  had  the  opportunity  to  plan 
and  conduct  development  programs  knowing  that  future  warranty 
requirements  were  a possibility. 

In  response  to  inquiries  about  their  products'  technological 
status,  most  contractors  reported  that  they  believed  that  their 
equipment  represented  a new  technological  development  as  compared 
with  a product  improvement  over  a previous  model.  To  the  extent 


that  utilization  of  advanced  technology  can  be  related  to  techni- 
cal risk  and  subsequent  warranty  risk,  this  environment  can  be 
expected  to  support  risk-identification  and  minimization  activi- 
ties during  design  and  development.  It  was  also  found  that  most 
of  the  contractors  did  not  have  past  relevant  warranty  experience 
that  would  have  increased  their  perception  of  risks  associated 
with  these  agreements  and  obligations. 

The  environment  of  concern  over  the  uncertain  timing  and 
definition  of  warranty  requirements,  the  advanced  technological 
status  of  the  equipment,  and  the  new  contractual  commitments 
containing  future  financial  uncertainties,  was  reflected  in  the 
critical  areas  of  warranty  negotiation.  The  definition  of 
relevant  failures,  measurement  of  field  reliability,  warranty 
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price,  and  turn-around  time  proved  to  be  the  most  difficult 
areas  for  final  warranty  negotiation. 1 

The  environment  for  warranty  application  was,  therefore, 
found  to  be  contractually,  technically,  and  financially  un- 
certain. This  uncertainty  will  have  to  be  considered  when 
warranty  impacts  are  identified  and  analyzed. 


JMany  of  the  critical  areas  overlap  or  are  interrelated  and  bear  heavily 
on  negotiation  difficulties  with  warranty  price. 


ho 


IV 


THE  IMPACT  OF  WARRANTIES  ON  CANDIDATE  SUBSYSTEM 
DEVELOPMENT  PROGRAMS 

The  previous  chapter  documented  the  application  of  war- 
ranties to  the  candidate  development  programs.  Based  upon  the 
data  derived  from  contractor  interviews  and  the  contractor- 
completed  questionnaires,  this  chapter  examines  the  impact  of 
the  warranties  on  the  conduct  of  the  candidate  development 
programs.  Section  A analyzes  the  reported  impact  in  terms  of 
the  functional  elements  of  a standardized  electronics  develop- 
ment program.  Section  B analyzes  the  warranty  requirements 
that  drove  the  reported  program  impacts.  And  Section  C analyzes 
the  barriers  associated  with  warranties  that  prevent  greater 
program  impacts  from  occurring. 

A.  DEVELOPMENT  PROGRAM  ELEMENT  IMPACTS 
1 . Summary  of  Questionnaire  Responses 

The  IDA  research  questionnaire1  identified  fifty-two 
functional  elements  of  a typical  electronics  development 
program.  These  elements  were  organized  into  thirteen  program 
categories,  each  of  which  encompasses  from  one  to  ten  func- 
tional elements.  The  impact  of  warranties  on  development 
program  elements  and  categories  was  recorded  for  the  total 
sample  and  for  various  subsets. 

The  actual  warranty  impact  on  candidate  program  elements 
was  derived  from  contractor  responses  to  the  questionnaire. 
Contractors  were  asked  to  identify  those  functional  program 
elements  that  were  actually  influenced  by  the  existence  or 
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possibility  of  a warranty  option  and  to  estimate  that  impact  in 
terms  of  incremental  effort  or  program  scope  change.  The  as- 
sumption was  that  the  warranty  was  to  be  a priced  option  and 
that  the  usual  Government  development  requirements  relating  to 
organic  maintenance  were  also  imposed  on  the  program.  The 
choice  of  response  for  each  functional  element  was  divided  into 
six  possibilities,  ranging  from  a moderate  decrease  in  effort  to 
a significant  increase  in  total  effort.  The  contractors  were  to 
assume  that  a "small"  increase  or  decrease  represented  approxi- 
mately a 10  percent  change,  a "moderate"  increase  or  decrease 
a 20  percent  change,  and  a "significant"  increase  represented 
anything  over  25  percent  additional  effort.  Table  4 presents 
a summary  of  the  contractor  estimates  (sixteen  in  the  total 
sample)  of  incremental  effort  required  in  each  of  the  develop- 
ment program  functional  areas  as  a result  of  the  existence  or 
possibility  of  a warranty  commitment. 

As  shown  in  the  table,  the  majority  of  the  responses  for 
each  functional  element  indicated  that  there  had  been  no  warranty- 
induced  changes  in  the  development  programs.  As  an  aggregate 
percentage,  two-thirds  of  the  responses  were  void  of  change  im- 
pacts, and  a total  of  83  percent  of  the  responses  indicated  an 
impact  of  10  percent  or  less. 

The  raw  data  from  the  questionnaire  responses  were  further 
processed  to  enable  the  investigators  to  derive  a quantitative 
measure  of  development  program  impact.  From  this  analysis,  the 
program  elements  receiving  the  greatest  impact  could  be  identi- 
fied and  ordered  by  decreasing  (or  increasing)  impact  magnitude.1 
The  data  processing  consisted  of  a valuation  technique  whereby. 


JA  difficulty  inherent  in  this  analytical  approach  was  recognized  to  be  the 
non-homogeneity  of  the  program  element  weights  among  the  candidate  programs, 
as  well  as  the  relative  weights,  measured  by  cost  or  effort  expended,  be- 
tween various  major  and  minor  program  elements  within  any  one  program.  The 
possibilities  of  these  sources  of  potential  bias  were  considered.  Ihe 
reader  should  recognize  this  limitation  whenever  extrapolations  of  these 
findings  are  made  to  other,  non-typical  programs. 
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Table  4.  ACTUAL  RIW  IMPACT  ON  CANDIDATE  DEVELOPMENT  PROGRAMS 


Development 
Program  Elements 


DESIGN 

Circuit 

Card  and  Board 

Module 

SRA 

LRU  or  FLU 
System  Architecture 
Built-In  Test 
Test  Equipment 
Ground  Support  Equipment 
Production  Tooling 
ANALYSIS 
Thermal 
Stress 
EMC 

Environment 
Fail ure-Mode- Effects 
Reliability 
Maintainability 
Producibllity 
Optimum  Repair  Level 
Cost-Performance 
MATERIALS  AND  PURCHASED  PARTS 
Part  Selection 
Vendor  Screening 
Cost  Estimating  and  Pricing 
PROTOTYPE  MANUFACTURE 
Component  Procurement 
Card  or  Board  Assembly 
Module  Assembly 
Box  or  Chassis  Assembly 
Functional  Test  and  Checkout 
EQUIPMENT  TEST 
Part/Component  Accept.  Test 
Breadboard  Tests 
Brassboard  Tests 
Equip.  Performance  Demo. 
Equipment  Environmental  Tests 
Equipment  Reliability  Tests 
QUALITY  ASSURANCE 
Quality  Engineering 
Inspection 

In-Process  Monitoring 
Test  Monitoring 
PROGRAM  MANAGEMENT 
Cost  Tracking 
FINANCIAL  MANAGEMENT 
Production  Cost  Analysis 
Economic  Escalation  Analysis 
Life-Cycle  Cost  Analysis 
Program  Financial  Control 
PRODUCT  SUPPORT 
Spares  Requirements 
Container  Design  and  Analysis 
STE/AGE  Design  and  Analysis 
Loqistlcs  Planning 
PRODUCTION  ENGINEERING 
Production  Analysis 
Manufacturing  Analysis 
VALUE  ENGINEERING 
DATA  AND  REPORTS 
CONTRACT  MANAGEMENT 


Moderate 

Decrease 


Number  of  Contractor  Responses 


Small 

Decrease 


No 

Change 

Small 

Increase 

Moderate 

Increase 

11 

3 

2 

10 

3 

2 

11 

3 

1 

11 

3 

1 

10 

3 

3 

11 

2 

2 

10 

3 

1 

10 

2 

1 

13 

1 

based  upon  the  guidelines  given  to  the  contractors,  a small 
Increase/decrease  was  assigned  a value  of  +10/-10,  a moderate 
increase/decrease  was  assigned  a Vulue  of  +20/-20,  a significant 
(greater  than  25  percent)  increase  was  arbitrarily  assigned  a 
value  of  +30,  and  no  change  was  given  a value  of  0.  1 

Table  5 presents  the  results  of  the  weighted  valuation  of 
the  program  elements.  This  processing  results  in  a measured 
average  impact  for  the  aggregate  programs  of  +5.^8  percent  with- 
in a range  of  0 to  19.60  percent.  The  average  increase  for  a 
typical  program  effort  was,  therefore,  estimated  by  the  con- 
tractors to  be  about  5.5  percent,  with  individual  programs  receiv- 
ing from  0 percent  to  almost  a 20  percent  increase  in  effort. 

The  weighting  system  also  illustrates  those  individual  program 
elements  and  groupings  of  program  elements  that  are  responsible 
for  the  major  changes  in  development  program  effort. 

2.  Analysis  of  Impacts  on  Program  Elements 

a . Des i qn 

The  design  task  of  the  development  program  was  anticipated 
to  be  an  area  in  which  a major  impact  would  be  recorded.  How- 
ever, the  data  show  that  the  average  impact  reported  for  this 
category  was  only  5 percent.  Seven  contractors  reported  no 
design  impact  whatever,  and  two  contractors  reported  only  a 
small  impact.  Only  three  contractors  estimated  greater  than  a 
10  percent  increase  in  the  design  effort;  however  one  of  those 
contractors.  Contractor  B,  felt  that  the  warranty  option  caused 
a 2h  percent  increase  in  design  effort.  The  greatest  average 
impacts  within  the  design  function  were  reported  in  areas  of 
test  equipment  design,  built-in  test  design,  and  card  and  board 
design. 2 


JIf  a value  of  50  were  assigned  instead  of  30,  the  results  of  the  responses 
would  be  altered  by  one  percentage  point. 

2Note  the  correlation  with  concerns  during  negotiations  about  identifying 
and  isolating  failures  at  the  organizational  level. 


♦Italicized  numbers  for  each  major  heading  are  totals  of  subelements. 

♦♦The  average  Impact  is  based  on  the  average  of  all  subelements.  Because  of  rounding,  the  average  of  the  major 
element  totals  is  5.54  percent. 


b.  Analysis 

The  analysis  portion  of  the  development  program  was  also 
expected  to  register  a large  warranty  impact.  However,  the  re- 
ported results  were  similar  to  those  recorded  for  design. 

Of  the  sixteen  contractors,  six  reported  no  impact  on  the 
analysis  function.  The  highest  impacts  approached  25  percent, 
as  with  the  design  functions.  Typically,  contractors  who  experi- 
enced an  impact  in  analysis  also  reported  an  impact  in  design. 

It  was  not  surprising  that  the  areas  of  greatest  impact  were 
reliability  analysis  and  maintainability  analysis.  Thermal 
analysis,  failure-mode-effects  analysis,  and  optimum  repair 
level  analysis  also  showed  significant  impacts.  In  most  cases, 
the  analysis  impact  was  greater  than  the  design  impact,  averag- 
ing  6.9  percent  vis-a-vis  5-0  percent  for  design. 


c.  Materials  and  Purchased  Parts 


This  area  of  the  development  program  also  was  reported  to 
have  experienced  a higher-than-average  impact  of  6.25  percent. 
These  findings  were  not  surprising  (except  for  the  generally 
low  level),  since  much  of  the  reliability  demonstrated  by  the 
space  program  has  been  claimed  to  have  been  derived  from  improved 
parts  and  materials  control.  It  was  anticipated  that  concern 
over  future  equipment  reliability  would  be  revealed  strongly  in 
the  materials  or  purchased  parts  specifications. 


d.  Prototype  Manufacture 

This  program  area  showed  a very  small  impact  (approximately 
a 2 percent  average  increase);  eight  contractors  reported  no 
impact.  Prom  this  response,  it  is  obvious  that  despite  the 
contractor's  desire  for  more  prototype  validation,  additional 
prototypes  were  not  being  built  in  the  current  programs  as  a 
result  of  warranty  considerations. 
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e . Equipment  Test 

Half  of  the  sixteen  contractors  indicated  no  impact  in  this 
program  area;  however,  two  contractors  reported  maximum  impacts 
("significant  increase")  for  many  of  the  equipment  test  elements. 

The  average  impact  on  the  equipment  test  function  was  approxi- 
mately a 5 percent  increase;  the  highest  impact  was  reported  for 
reliability  testing. 

f . Quality  Assurance 

This  was  another  area  in  which  additional  effort  resulting 
from  warranty  considerations  was  expected  to  be  evident.  How- 
ever, as  Table  5 illustrates,  an  aggregate  impact  of  only  4.8 
percent  was  reported.  Half  of  the  contractors  reported  no 
additional  effort  in  this  category. 

J 

g . Program  Management 

Program  management  similarly  received  added  emphasis  from 
only  about  half  of  the  reporting  contractors  (seven).  However, 
the  impact  reported  by  those  who  were  affected  was  split  between 
a moderate  and  a small  effect,  resulting  in  a group  average  of 
those  reporting  an  impact  of  17  percent.  The  overall  sample 
impact  (counting  those  reporting  no  impact)  was  7-5  percent. 

h . Financial  Management 

Although  half  of  the  sixteen  contractors  indicated  no 
impact  on  financial  management,  the  range  of  impacts  went  up 
to  20  percent  for  two  of  the  contractors.  Life-cycle  cost 
analysis  showed  a 10  percent  increase  in  impact  and  production 
cost  analysis  showed  a 7-5  percent  increase. 

i . Product  Support 

Seven  contractors  reported  no  impact  on  product  support, 
but  one  contractor  reported  a 20  percent  increase.  There  was 
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almost  a 12  percent  average  Increase  In  effort  in  the  logistics 
planning  function. 

j . Other  Program  Elements 

Other  program  elements  examined  for  a warranty  impact  were 
production  engineering,  value  engineering,  data  and  reports,  and 
contract  management.  Data  and  reports  and  contract  management 
were  the  areas  of  highest  reported  impact  with  6.9  and  7.5  per- 
cent, respectively.  It  should  be  noted  that  these  two  elements 
are  not  direct  contributors  to  equipment  design  reliability. 

3.  Combined  Development  Program  Impact 

Figure  6 is  a histogram  showing  the  average  warranty  impact 
for  the  thirteen  major  categories  into  which  the  functional  pro- 
gram elements  were  grouped.  As  discussed  above,  the  highest 
impacts  were  reported  in  the  areas  of  program  management, 
financial  management,  contract  management,  analysis,  and  data 
and  reports. 

Because  many  contractors  reported  no  measurable  impact  in 
many  program  elements,  an  analysis  was  made  to  quantify  the 
program  element  impact  based  upon  only  those  contractors  who 
reported  an  impact.1  The  results  of  this  analysis  are  compared 
with  the  full  aggregate  sample  analysis  in  Table  6 below. 

The  results  presented  in  Table  6 demonstrate  that,  while 
the  aggregate  impact  for  all  contractors  responding  to  the 
questionnaire  was  about  5 percent,  those  program  elements  that 
received  a warranty  impact  other  than  zero  combined  to  give  an 


!No  correlation  could  be  found  between  the  contractor  impact  response  and 
equipment  application,  Service  management,  or  equipment  type.  For  example, 
it  was  expected  that  contractors  who  were  merely  developing  a "product 
improvement"  would  report  a smaller  impact  than  the  others.  However,  three 
of  the  four  contractors  who  characterized  their  equipment  as  a "product 
improvement"  (K,  N and  H)  reported  impacts  in  at  least  nine  of  the  thirteen 
program  categories.  A full  discussion  of  these  results  and  analyses  is 
presented  in  Chapter  V. 
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average  development  program  impact  of  12.4  percent.  The  program 
categories  also  change  in  relative  importance.  In  the  total 
aggregate  sample,  the  highest  impacts  were  reported  in  program 
management,  contract  management,  analysis,  data  and  reports, 
and  financial  management.  The  major  impacts  for  Just  those 
elements  receiving  impacts  were  also  observed  to  include  program 
management,  financial  management,  and  contract  management.  How- 
ever, data  and  reports  now  has  the  highest  reported  impact  at 
18.33  percent,  and  production  engineering,  value  engineering, 
and  materials/purchased  parts  have  been  added  to  financial 
management  as  leading  areas  of  warranty  impact. 

Table  7 illustrates  the  reported  aggregate  impact  on  in- 
dividual development  program  elements  in  terms  of  an  ordered 
listing  by  impact  magnitude.  Note  the  relative  position  of  the 
design-related  program  elements  (identified  with  an  asterisk), 
which  indicates  that  design  efforts  are  being  affected  by  the 
warranty  environment. 

B.  WARRANTY  REQUIREMENTS  DRIVING  PROGRAM  IMPACTS 

While  this  subject  area  was  not  explicitly  addressed  in  the 
research  questionnaire,  in  our  interviews  with  contractors  we 
attempted  to  identify  some  of  the  principal  warranty  requirements 
responsible  for  the  reported  development  program  impacts.  We 
found  that  most  requirements  could  be  categorized  into  one  of 
two  conceptual  classes.  The  first  class  consists  of  those 
requirements  that  are  contractually  imposed  ani  that  affect 
equipment  features  or  development  or  production  program  content. 

Some  of  the  requirements  identified  in  this  class  are 
listed  below: 

• Display  of  warranty  information  on  equipment 

• Labels  on  unit  for  recording  installation  and  removal 
date 

• Seals 
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Table  7.  RELATIVE  IMPACT  UPON  INDIVIDUAL 
PROGRAM  ELEMENTS 


Development  Program  Elements 

Percent  Change 
in  Development 
Program  Effort 

Reliability  Analysis* 

12.50 

Logistics  Planning 

11.85 

Life-Cycle  Cost  Analysis 

10.00 

Maintainability  Analysis* 

10.00 

Equipment  Reliability  Tests 

8.13 

Failure-Mode-Effects  Analysis* 

8.13 

Test  Equipment  Design 

8.13 

Thermal  Analysis* 

8.13 

Optimum  Repair  Level  Analysis 

8.13 

Production  Cost  Analysis 

7.50 

Cost  Tracking 

7.50 

Contract  Management 

7.50 

Test  Monitoring 

7.50 

BITE  Design* 

6.88 

Environmental  Analysis* 

6.88 

Part  Selection* 

6.88 

Vendor  Screening 

6.88 

Data  and  Reports 

6.88 

Card/Board  Design* 

6.25 

Equipment  Performance  Demo.  Test 

6.25 

STE/AGE  Design  & Analysis* 

6.25 

Equipment  Environmental  Tests 

5.63 

System  Architecture  Design* 

5.63 

LRU/FLU  Design* 

5.63 

Module  Design* 

5.00 

SRA  Design* 

5.00 

Cost  Estimating  & Pricing 

5.00 

Program  Financial  Control 

5.00 

Cost-Performance  Analysis 

5.00 

Brassboard  Tests 

5.00 

Economic  Escalation  Analysis 

4.38 

Quality  Engineering 

4.38 

Produclbility  Analysis 

4.38 

Circuit  Design* 

4.38 

Spares  Requirements 

4.38 

Producibility  Analysis  (Production  Engr'g.)4.38  j 

Stress  Analysis* 

4.38 

Production  Tooling  Design 

4.38 

Quality  Assurance  Inspection 

3.75 

Manufacturing  Analysis 

3.75 

Functional  Test/Checkout  Proto.  Man.  3.75 

In-Process  Quality  Assur.  Monitoring  3.75 

Part/ Component  Accept.  Tests 

3.75 

Container  Design  & Analysis 

3.13 

Breadboard  Tests 

3.13 

Value  Engineering 

2.50 

Component  Procurement-Proto.  Man. 

2.50 

Module  Assembly-Proto.  Man. 

1.88 

EMC  Analysis 

1.88 

Box/Chassis  Assy. -Proto.  Man. 

1.25 

Card/Board  Assy. -Proto.  Man. 

1.25 

Ground  Support  Equip.  Design 

-1.25 

AVERAGE 

5.48 

♦Equipment  Design-Related  Program  Elements 
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• Elapsed-time  indicators 

• Data  and  reports 

• ECP  and  configuration  control  procedures 

• Bonded  storerooms 

• Warranty  repair  facility. 

The  second  class  of  warranty  requirements  consists  of  speci- 
fied equipment  or  contractor  performance  parameters  that  directly 
influence  the  achieved  equipment  reliability,  maintainability, 
or  availability.  These  also  are  variables  that  can  be  directly 
influenced  by  contractor  trade-offs  during  the  design  and 
development  process.  Examples  of  this  class  of  warranty 
requirements  include: 

• Guaranteed  MTBF  (mean  time  between  failures) 

• RTOK  (retest)  OK) 

• Average  MTTR  (mean  time  to  repair) 


• CTR  (cost  to  repair) 

• Agreed  TAT  (turn-around  time)  without  penalties 

• ECP  costs. 

The  requirements  in  the  second  class  act  in  concert  such 
that  the  classical  trade-off  between  MTBF  and  MTTR  is  further 
complicated  by  cost  penalties  associated  with  required  TATs  and 
a desire  to  minimize  the  number  of  RTOKs . These  requirement 
considerations,  for  example,  could  be  expected  to  lead  to  in- 
creased effort  on  many  of  the  development  program  elements.  In 
the  general  category  of  design,  we  could  expect  increased  effort 
on  system  architecture  to  investigate  various  circuit  partition- 


often  needed  to  achieve  the  appropriate  trade-offs.  The  MTBF 
could  also  be  increased  by  improving  inherent  reliability  through 
upgraded  parts  or  more  vendor  screening.  An  increased  effort  in 
prototype  manufacturing  and  equipment  test  might  lead  to  better 
information  on  such  parameters  as  MTBF,  MTTR,  and  CTR.  Efforts 
in  product  support  definitely  affect  the  MTTR  and  TAT  parameters. 
Efforts  in  production  engineering  can  indirectly  affect  the 
achieved  field  reliability  of  the  equipments.  Finally,  other 
program  impacts  of  a financial,  contractual,  or  managerial 
nature  can  stem  from  the  increased  levels  of  direct  program 
activity. 

The  two  classes  of  warranty  requirements  also  interact. 

For  example,  seals  and  elapsed-time  indicators  must  be  analyzed, 
designed,  purchased  or  manufactured,  and  tested.  And  bonded 
storerooms  and  contractor  repair  facilities  must  be  planned 
along  with  their  appropriate  spares  inventories  and  test 
equipment . 

It  can  be  seen  from  the  above  discussion  that  although 
there  can  be  many  impacts  stemming  from  the  warranty  require- 
ments, the  specific  cause-effect  relationships  are  lost  in  over- 
lapping influences.  Nevertheless,  it  is  clear  that  the  warranty 
requirements  are  collectively  responsible  for  numerous  impacts 
on  the  candidate  development  programs. 

C.  IMPACT  BARRIERS 

Our  initial  exploratory  interviews  with  the  contractors 
convinced  us  that,  while  marginal  impacts  occurred  as  a result 
of  current  or  recent  development  program  warranty  requirements, 
the  potential  for  more  significant  impacts  was  very  great. 

These  beneficial  impacts,  in  terms  of  emphasizing  reliability 
and  maintainability  during  design  and  development,  were  being 
inhibited  or  prevented  by  program  variables  and  contract  require- 
ments that  effectively  prevented  contractors  from  taking  those 
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actions  they  believed  to  be  in  the  best  interests  of  both 
parties.  Some  of  the  basic  barriers  were  identified  during  the 
initial  interviews  and  were  used  to  structure  the  questionnaire. 
Other  barriers  were  identified  by  the  responses  to  specific 
questions  in  the  questionnaire.1 

1 . Primary  Barriers 

The  primary  barriers  identified  during  the  initial  inter- 
views were — 

(a)  The  uncertainty  of  future  warranty  option  terms,  condi- 
tions, and  other  details  during  engineering  development. 

(b)  The  uncertainty  associated  with  the  option  status  of 
the  warranty. 

(c)  Product  and  process  specification  rigidities  that 
prevented  contractor  design  and  development  program 
flexibility. 

In  several  development  programs,  the  specific  terms  and 
conditions  of  the  warranty  had  not  been  defined  even  though  the 
programs  were  well  into  engineering  development.  Because  of 
this  uncertainty,  the  contractor  lacked  the  motivation  to 
institute  signficant  changes  in  design  and  development.  Con- 
tractors reported  that  had  these  requirements  been  known 
earlier,  the  warranty  impacts  on  the  design  and  development 
program  could  have  been  greater. 

Contractors  also  reported  that  the  option  status  of  warran- 
ties was  perhaps  the  most  signficant  barrier  to  increased  war- 
ranty impacts.  The  uncertainty  as  to  whether  or  not  the 
Government  would  eventually  exercise  the  option  led  to  compro- 
mises in  equipment  design  and  to  doubts  concerning  the  Govern- 
ment’s credibility  for  warranty  application.  The  basic 


1See  questions  no.  2 and  page  D-12  and  no.  3 on  page  D-13  of  the  question- 
naire, Appendix  D. 


requirements  of  the  contract  still  required  a design  for 
organic  maintenance — any  changes  to  the  design  to  enhance 
contractor  maintenance  were  therefore  considered  cautiously. 

Contractors  also  indicated  that  there  were  conflicts 
between  the  objectives  of  warranties  and  the  application  of 
some  military  specifications  to  their  equipment.  They  reported 
that  the  inherent  guarantee  of  the  warranty  superseded  the 
requirement  for  many  of  the  usual  equipment  design  and  process 
specifications.  They  stated  that  if  they  were  allowed  greater 
freedom  to  design  and  develop  the  equipment,  development  program 
impacts  resulting  from  warranty  requirements  would  be  much 
higher. 

These  three  primary  barriers  were  judged  to  be  fundamental 
determinants  of  warranty  impact  and  were  established  as  major 
conditions  for  potential  impact  analysis.  The  effects  of  removing 
these  barriers  were  subsequently  investigated  in  the  questionnaire. 
The  results  of  that  investigation  are  presented  in  Chapter  V. 

2.  Additional  Barriers 

Table  8 presents  the  results  of  our  inquiry  into  additional 
barriers  that  had  not  been  surfaced  in  the  initial  contractor 
interviews.  It  is  apparent  from  the  reported  responses  that 
there  are  four  other  major  constraints  to  warranty  impacts. 

These  are  (1)  development  funding,  (2)  development  schedule, 

(3)  unit-production  cost  goal,  and  (4)  performance  specifica- 
tions. In  general,  these  variables  represent  the  primary 
degrees  of  freedom  available  to  the  development  contractors. 

Discussions  with  contractor  personnel  revealed  that  the 
development  programs  for  their  electronics  subsystems  continue 
to  be  time  and  funding  constrained. 1 Programs  are  planned  to 


lrrhis  was  also  true  for  other  programs  and  contractors  investigated  during 
previous  IDA  Desigi-to-Cost  research. 
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I 

[match  major  prime  equipment  delivery  schedules  and  total  funds 
available.  Design  iterations  and  additional  test  programs  to 
give  contractors  greater  confidence  in  their  product  performance 
are  casualties  of  forces  exerted  by  time,  funds,  and  marketing 
pressures.1  Unit-production  goals,  set  prior  to  development, 
also  present  barriers  to  changes  in  design  that  would  enhance 
reliability.  This  is  particularly  the  case  when  the  introduc- 
tion of  warranty  considerations  occurs  after  the  preliminary 

; 

design  reviews  have  been  accomplished.  At  this  point,  the 

■■ 

design  may  already  be  optimized  for  production  cost  rather  than 
reliability  assurance. 

Table  8.  BARRIERS  TO  DESIGN  AND  DEVELOPMENT  IMPACT 


Barrier  Candidates 

Frequency  of  Response 

Minor  Constraint 

Major  Constraint 

Total 

Development  Funding 

3 

13 

16 

Development  Schedule 

5 

9 

14 

Unit  Production  Cost  Goal 

6 

7 

13 

Performance  Specifications 

9 

2 

11 

Reliability  Goals 

5 

2 

7 

Reprocurement  Data  Rights 

3 

4 

7 

Source  Selection  Criteria 

4 

3 

7 

Part  and  Process  Specifications 

4 

2 

6 

Report  Requirements 

2 

1 

3 

Handbook  Changes 

0 

2 

2 

Reliability  Demonstration 

1 

0 

1 

Physical  Design 

0 

1 

1 

Warranty  Terms  & Conditions 

0 

1 

1 

"Traditionalism"  Attitude 

0 

1 

1 

Support  Equip.  Changes 

0 

1 

1 

Organic  Maintenance  Requirement 

0 

1 

1 

Limited  Field  Testing 

0 

1 

1 

lton  important  point.  No  competitive  contractor  is  very  willing  to  admit 
that  he  needs  more  time  and  money  to  gain  confidence  that  he  can  meet  • 
(guarantee)  the  performance  specifications  for  the  candidate  equipment. 
Thus,  quite  often  it  was  observed  that  these  activities  were  performed 
using  company  funds  when  the  contractor  believed  that  the  warranty  would 
actually  be  implemented. 
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The  contractors  were  also  asked  to  Identify  the  most  criti- 
cal barrier  from  among  their  responses  shown  in  Table  8.  Their 
assessment  follows  the  pattern  established  by  the  frequency  of 
response,  as  illustrated  in  Table  9«  The  contractors  believed 
that  their  programs  needed  more  effort  (in  areas  we  will  sub- 
sequently identify)  to  ensure  a greater  probability  of  achieving 
the  reliability  guarantees.  This  effort  would  require  more 
money  and  development  time.  Performance  and  unit-production 
cost  goal  requirements  presented  additional  barriers,  which 
were  viewed  as  being  relatively  independent  of  the  funding  and 
schedule  restrictions.1 


Table  9.  BARRIER  RESPONSE  ANALYSIS 


Reported  Barrier 

Ranking  by 
Frequency 

Ranking  by 
Importance 
to  Contractors 

Development  Funding 

1 

1 

Development  Schedule 

2 

2 

Unit-Production  Cost  Goal 

3 

4 

Performance  Specifications 

4 

3 

D.  FINDINGS 

The  analysis  of  the  contractor  interviews  and  the  question- 
naire responses  led  to  the  following  major  findings  concerning 
the  impact  of  warranties  on  the  current  candidate  development 
programs . 


• • v estimated  unit-production  cost  difference  between  a 

- - • t<  meet  reliability  specifications  and  a design 

t,  nynpt  reliability  specifications. 


The  reported  impact  of  warranty  considerations  on  the  can- 
didate programs  was,  on  the  average,  small — the  aggregate  im- 
pact was  a 5.5  percent  increase  in  overall  effort.  For  the 
total  sample,  the  greatest  impacts  were  recorded  in  the  func- 
tional areas  of  program,  financial,  and  contract  management, 
analysis,  and  data  and  reports. 

An  analysis  of  individual  candidate  programs  and  program 
elements  indicated  a dual  impact  distribution;  approximately 
half  of  the  sample  indicated  some  impact.  Those  contractors 
reporting  an  impact  indicated  an  average  increase  in  program 
effort  of  12.4  percent.  These  impacts  also  were  reported  in 
the  functional  areas  of  data  and  reports  and  contract,  program 
and  financial  management. 

2 . Requirements  Driving  Program  Impacts 

A number  of  warranty  requirements  were  identified  as  driv- 
ing the  program  impacts.  Those  requirements  were  divided  into 
two  groups:  those  relatively  fixed  because  of  contractual 

requirements;  and  those  that  directly  influence  the  reliability 
and  maintainability  parameters  of  the  equipment. 

3.  Barriers  to  Program  Impacts 

Many  important  barriers  to  changes  in  the  equipment  design 
or  the  development  programs  were  identified  during  our  investi- 
gations. Primary  barriers  were  (a)  uncertainty  of  warranty 
requirements,  (b)  uncertainty  of  warranty  option  decisions, 
and  (c)  rigid  product  and  process  specifications.  Additional 
barriers  reported  by  the  contractors  were  lack  of  adequate 
development  funds,  constrained  development  schedules,  conflict- 
ing unit-production  cost  and  reliability  goals,  and  performance 
specifications. 
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POTENTIAL  IMPACT  OF  WARRANTIES  ON  CANDIDATE 
DEVELOPMENT  PROGRAMS 

As  reported  in  Chapter  IV,  our  research  led  to  the  identi- 
fication of  three  primary  barriers  to  achieving  beneficial  design 
and  development  program  impacts.  Accordingly,  this  chapter 
examines  the  potential  warranty  impacts  if  the  three  primary 
barriers  were  removed. 

The  first  condition  examined  is  the  situation  in  which 
the  terms  and  conditions  of  the  warranty  option  have  been 
negotiated  (except  for  price)  at  the  beginning  of  the  engineer- 
ing development  program.  The  second  condition  eliminates  the 
option  status  from  the  first  condit ion--it  is  known  at  the  out- 
set of  engineering  development  that  the  equipment  will  even- 
tually be  procured  with  a warranty  and  that  Service  organic 
maintenance  will  not  be  implemented,  at  least  for  several  years 
after  production  deliveries  begin.  The  third  condition  in- 
volves the  additional  feature  of  specification  relief — e.g., 
all  design,  performance,  and  process  specifications  and  require- 
ments are  flexible  and  negotiable  except  for  critical  perform- 
ance, cost,  and  reliability  requirements.1 

A.  POTENTIAL  PROGRAM  IMPACTS  AND  WARRANTY  DECISIONMAKING 

This  section  compares  the  actual  estimated  development 
program  impacts  reported  by  the  contractors  for  their  current 
programs  with  two  alternate  conditions:  (1)  the  impact  if 

warranty  terms  and  conditions  were  known  at  the  beginning  of 


lrThis  condition  is  similar  to  that  which  is  conceptually  planned  for  design- 
to-cost  acquisitions. 


engineering  development,  and  (2)  the  impact  if  the  warranty 
were  a firm  contractual  commitment.1 

1 . Major  Program  Categories 

Tables  10  and  11  present  the  weighted  warranty  impacts,  as 
compiled  from  the  contractor  questionnaires,  for  the  beginning 
and  firm  conditions.2 

The  average  impact,  weighted  over  all  development  program 
categories  and  measured  in  terms  of  a percentage  increase  in 
effort,  climbs  from  a value  of  approximately  5-5  percent  for 
the  actual  warranty  conditions  examined  in  Chapter  IV  to  an 
estimated  value  of  8.2  percent  if  the  terms  and  conditions  were 
known  at  the  beginning  of  engineering  development,  and  to  a 
value  of  approximately  10.5  percent  if  the  RIW  requirements 
were  not  an  option  but  firm. 

The  aggregate  impact  ratio  of  beginning  to  actual  is 
approximately  1.5  (8. 2:5. 5);  the  ratio  of  firm  to  beginning 
is  approximately  1.3  (10.5:8.2).  In  other  words,  the  esti- 
mated increase  in  impact  under  the  beginning  conditions  is  50 
percent  greater  than  the  actual  reported  increase  in  impacts. 
Further,  the  firm  condition  impact  is  another  30  percent  greater. 
The  firm  condition  impact  is  thus  90  percent  greater  than  the 
actual  impact  (10.5:5-5  = 1-9).  The  development  program  impact 
leverage  that  can  then  be  attributed  to  these  conditions  is 
thought  to  be  significant.3 


3For  discussion  purposes,  the  three  warranty  requirement  conditions  de- 
scribed in  this  paragraph  will  be  referred  to  in  abbreviated  form  as  actual , 
beginning , and  firm. 

2Table  5 (Chapter  IV)  presented  similar  data  for  the  warranty  impact  actually 
experienced  during  the  candidate  programs. 

3The  term  "leverage"  is  used  throughout  this  chapter  to  refer  to  the  rela- 
tive increase  in  impact  or  impact  multiplier  that  is  attributable  to 
warranty- related  decisions  made  earlier  in  the  acquisition  process. 
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Table  10.  WARRANTY  IMPACT  IF  OPTION  TERMS  AND  CONDITIONS 
WERE  KNOWN  AT  BEGINNING  OF  ENGINEERING  DEVELOPMENT 


•Italicized  numbers  for  each  major  heading  are  totals  of  subelements. 

**The  average  Impact  Is  based  on  the  average  of  all  subelements.  Because  of  rounding,  the  average  of  the  major 
element  totals  Is  8.17  percent. 
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Table  11.  WARRANTY  IMPACT  IF  WARRANTY  WERE  FIRM  AND 

NOT  AN  OPTION 


•Contractor  J did  not  respond  to  this  condition  because  he  felt  It  was  not  applicable  to  his  development  program.  Therefore 
the  averages  presented  are  based  on  15  contractors. 

•♦Italicized  numbers  for  each  major  heading  are  totals  of  subelements. 

***The  average  impact  Is  based  on  the  average  of  all  subelements.  Because  of  rounding,  the  average  of  the  major  element 
totals  Is  10.38  percent. 
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Table  12  provides  a ranking  of  the  above  multiplier  ratios 
for  each  of  the  major  development  program  categories.  The 
following  sections  discuss  the  individual  development  program 
categories  and  their  constituent  subelements. 

2.  Desi gn 

Figure  7 presents  a histogram  illustrating  the  impact  of 
warranties  on  the  design  program  elements  under  the  three 
conditlons--actual , beginning , and  firm. 

Table  13  presents  the  leverage  ratios  under  the  three 
conditions  for  the  total  design  group  and  for  those  elements 
on  which  a high  degree  of  leverage  is  obtained. 

Table  13.  IMPACT  LEVERAGE  UPON  DESIGN 


Design  Category 

Beginning/ 

Actual 

Firm/ 

Beginning 

Firm/ 

Actual 

Average 

1.  59 

1.39 

2.  20 

Module 

2.00 

1.40 

2.80 

SRA 

1.88 

1.42 

2.67 

Circuit 

1.57 

1 .45 

2.28 

System  Architecture 

1.67 

1.14 

1.90 

Test  Equipment 

1.31 

1.44 

1.89 

The  areas  of  greatest  design  impact  reported  by  the  con- 
tractors under  the  beginning  and  firm  conditions  are  largely  an 
extension  of  those  cited  in  the  actual  development  programs — 
namely,  BITE,  system  architecture,  sensors-indicators-protective 
circuits,  and  higher  inherent  design  reliability.  Many  examples 
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of  impact  mentioned  by  contractors  in  the  questionnaires  were 
in  the  area  of  BITE.  The  following  areas  were  also  mentioned: 
repackaging  of  the  system,  thermal  sensors,  automatic  test 
equipment,  circuit  partitioning,  and  module-card  design  tailor- 
ing to  enhance  contractor  repair. 

3 . Analysis 

Figure  8 presents  the  histogram  of  the  warranty  impacts 
for  the  functional  elements  in  the  analysis  category.  Table 
14  presents  the  leverage  ratios  for  the  total  analysis  group 
and  for  major  subelements. 


Table  14.  IMPACT  LEVERAGE  UPON  ANALYSIS 


Analysis  Category 

Beginning/Actual 

Firm/Beginning 

Firm/Actual 

Average 

1.34 

1.22 

2.63 

Cost  Performance 

1.50 

1.16 

1.73 

FMEA 

1.46 

1.12 

1.64 

Environment 

1.45 

1.20 

1.74 

Reliability 

1.45 

1.03 

1.49 

Producibility 

1.43 

1.49 

2.13 

EMC 

1.00 

2.84 

2.84 

Stress 

1.29 

1.54 

1.98 

t? 


Note  that  the  two  areas  (EMC  & stress)  that  show  the  highest 
firm /beginning  ratios  are  not  in  the  top  five  areas  that  had 
the  highest  beginning / actual  leverage.  This  may  indicate  that 
a point  of  diminishing  returns  has  been  reached  in  some  of  these 
areas.  Cost-performance  analysis  showed  the  highest  leverage  of 
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Figure  8.  SUMMARY  OF  WARRANTY  IMPACTS:  ANALYSIS 


the  beginning /actual  comparison,  which  indicates  the  considera- 
tion of  key  "early  performance-support  cost  trade-offs"  that 
had  not  been  previously  performed. 

4 . Materials/Purchased  Parts  and  Prototype  Manufacture 

Figure  9 presents  the  histogram  of  the  warranty  impact  for 
two  functional  areas,  materials/purchased  parts  and  prototype 
manufacture.  The  three  subelements  of  the  materials  and  pur- 
chased parts  group  show  a very  consistent  pattern  with  respect 
to  the  leverage  ratios.  The  area  of  impact  most  often  mentioned 
by  contractors  was  parts  selection,  e.g.,  an  increased  emphasis 
would  be  placed  on  component  selection  based  upon  company  tests 
and  long-term  warranty  considerations. 

Prototype  manufacturing  was  the  area  that  had  the  lowest 
overall  actual  impact.-  Nevertheless,  or  perhaps  because  of 
this,  it  has  the  highest  leverage.  For  example,  going  from 
the  actual  to  the  beginning  case  shows  an  88  percent  increase, 
going  further,  from  the  beginning  to  the  firm  case,  shows  an 
additional  67  percent  increase.  Contractors  seem  to  be  partic- 
ularly desirous  of  establishing  pilot  production  runs  and  test- 
ing those  pilot  production  units  under  field  conditions. 

5.  Financial  Management  and  Product  Support 

Figure  10  presents  the  histogram  of  warranty  impact  for  the 
financial  management  and  product  support  groups. 

In  the  financial  management  group,  economic  escalation 
analysis  shows  the  most  leverage  as  warranty  requirements  be- 
come more  certain.  In  the  product  support  group,  container 
design  and  analysis  shows  the  most  leverage. 

It  is  interesting  to  note  that  as  warranty  decisions  go 
from  the  beginning  case  to  a firm  warranty  (no  option),  three 


INCREASE  IN  EFFORT  (percent) 


y</f  Reported  impact  on  candidate  engineering  development  effort 

Potential  impact  if  terms  and  conditions  of  warranty  option  were  fixed  at  the  beginning  of  engineering  development 
\\\  Potential  impact  if  warranty  were  a firm  requirement  at  the  beginning  of  engineering  development 
*25-74-5 


Figure  9.  SUMMARY  OF  WARRANTY  IMPACTS:  MATERIALS/PURCHASED 

PARTS  AND  PROTOTYPE  MANUFACTURE 
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Figure  10.  SUMMARY  OF  WARRANTY  IMPACTS:  FINANCIAL  MANAGE- 

MENT AND  PRODUCT  SUPPORT 
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elements  decrease  in  effort — production-cost  analysis,  life- 
cycle  cost  analysis,  and  logistics  planning.  This  is  not  un- 
expected, since  there  are  fewer  alternatives  to  consider  if  the 
warranty  is  firm. 

6.  Equipment  Test  and  Quality  Assurance 

Figure  11  presents  the  warranty  impacts  in  the  equipment 
test  and  quality  assurance  categories  under  the  three  conditions. 

The  impact  on  equipment  reliability  demonstration  and 
verification  tests  goes  up  markedly  in  moving  from  the  actual 
to  the  beginning  cases  but  less  dramatically  for  the  firm  case. 

It  may  be  that  increased  reliability  testing  has  progressed 
almost  as  far  as  possible  in  the  beginning  case  and  there  is 
little  left  to  do  in  the  firm  case,  because  the  contractors 
typically  do  not  have  the  freedom  to  design  the  tests  the  way 
they  would  like.1  Also,  the  tests  may  have  been  expanded  as  much 
as  possible  to  accord  with  the  factory  maintenance  philosophy 
while  still  staying  within  required  test  specifications. 

7 . Miscellaneous  Areas 

To  complete  the  list  of  development  program  elements. 

Figure  12  presents  the  histogram  of  warranty  impacts  for  the 
following  areas: 

• Program  management  (cost  tracking) 

• Production  engineering 

• Producibility  analysis 

• Manufacturing  analysis 

• Value  engineering 

• Data  and  reports 

• Contract  management. 


10r  perhaps  the  development  program  funding  and  schedule  barriers  still  are 
constraints. 
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Cost  tracking  becomes  increasingly  more  important  as  the 
conditions  are  relaxed.  It  had  the  eleventh  highest  impact  in 
the  actual  case,  the  seventh  highest  in  the  beginning  case,  and 
the  fifth  highest  in  the  firm  case. 


Combined  Development  Program  Impacts 


Figure  13  summarizes  the  reported  warranty  impacts  under 
the  three  conditions  for  the  thirteen  major  functional  element 
categories.  From  the  figure,  it  can  be  seen  that  the  greatest 
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overall  average  Impact  for  the  firm  conditions  Is  registered  in 
program  management  (14  percent),  followed  by  data  and  reports 
(12  percent)  and  materials  and  purchased  parts  ^11. 75  percent). 

If  only  the  beginning  conditions  prevail,  the  greatest  impact 
is  reported  again  in  the  program  management  area  (10.6  percent), 
followed  by  financial  management  (10.3  percent)  and  contract 
management  (9.4  percent).  Note  that  the  last  two  are  indirect 
cost  elements. 

As  observed  in  Chapter  IV,  not  all  contractors  reported 
impacts  for  each  of  the  program  elements  as  the  barriers  were 
removed.  If  only  those  program  elements  that  received  an  impact 
are  considered,  the  aggregate  average  impact  is  correspondingly 
higher  in  each  category,  with  data  and  reports,  program  manage- 
ment, production  engineering,  and  materials  management  leading 
the  impact  estimates  when  the  warranty  is  firm.  A summary  of 
the  reported  impacts  for  both  the  aggregate  sample  and  the  sub- 
set of  program  elements  receiving  an  impact  is  presented  in 
Table  15. 1 As  shown  by  the  table,  those  contractors  who  re- 
ported impacts  (other  than  zero),  on  the  average,  did  not  dis- 
criminate between  the  actual  case  and  the  case  in  which  warranty 
requirements  are  known  at  the  beginning  of  engineering  development. 

One  reason  for  the  small  differences  in  sample  reported  im- 
pacts is  that,  for  five  programs,  there  was  no  difference2  in 
most  of  the  impacts  due  to  the  imposed  conditions.  For  two 
contractors,  the  systems  were  significantly  developed  and  tested 
prior  to  receipt  of  the  RFPs . The  other  contractors  either 
treated  the  option  as  a "firm  requirement"  and  proceeded  on  the 
basis  that  the  option  would  be  exercised  or  indicated  that,  "a 


lfIhe  authors  recognize  that  this  analysis  raises  two  difficult  research 
questions.  First,  should  a reported  inpact  of  zero  in  a program  element 
be  counted?  Second,  what  interpretation  should  be  made  for  the  subset 
consisting  of  only  the  program  elements  receiving  an  inpact  other  than  zero? 

2A11  questionnaire  entries  for  each  of  the  52  development  program  elements 
were  identical  for  the  actual , beginning , and  firm  cases. 
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Note:  Actual  = reported  impact  on  candidate  engineering  development  program. 

Beginning  = potential  Impact  if  terms  and  conditions  of  warranty  option  were  fixed  at  beginning  of 
engineering  development. 

Firm  = potential  Impact  If  warranty  were  firm  requirement  at  the  beginning  of  engineering  development. 
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responsible  contractor  cannot  distinguish  any  difference  between 
impact  of  a priced  option  and  a firm  obligation  since  a warranty 
commitment  cannot  be  economically  fulfilled  unless  considered 
during  initial  design." 

In  two  additional  programs,  the  change  in  impact  due  to  the 
imposed  conditions  was  small — in  both  cases  the  actual  impact 
was  equal  to  the  beginning , and  the  firm  case  registered  a small 
increase  over  both. 

In  the  remaining  nine  programs,  substantial  effects  were 
observed  as  a result  of  varying  the  conditions.  In  three 
development  programs,  for  example,  the  actual  impact  was  zero, 
but  it  was  estimated  that  there  would  have  been  an  8.1  percent, 
9.6  percent,  and  10.8  percent  increase  in  effort,  respectively, 
if  the  warranty  terms  and  conditions  had  been  known  at  the 
beginning  of  engineering  development. 

It  should  be  noted  that  the  actual  situation  in  the  six 
P-1 6 development  programs  was  that  the  RIW  terms  and  conditions 
were  known  at  the  beginning  of  engineering  development.1  The 
increase  in  the  total  impact  if  terms  and  conditions  had  not 
been  negotiated  at  the  beginning  of  engineering  development 
for  the  P-16  programs  is  likely  to  have  been  greater  than 
reported. 

Examining  the  ten  remaining  contractor  development  pro- 
grams not  associated  with  the  F-16  program,  the  increase  in 
impact  (going  from  actual  to  beginning ) approaches  112  percent. 
This  can  be  compared  with  the  50  percent  increase  in  impact 
observable  over  all  sixteen  programs,  as  reflected  on  the 
histograms . 

1 These  were  the  F-l6  avionics  development  programs,  which  were  negotiated 
under  subcontract  purchase  orders  with  General  Dynamics.  A key  question 
for  further  analysis  is  whether  these  programs  were  really  at  the  start 
of  engineering  development  or  whether  they,  in  fact,  were  essentially  at 
the  end. 
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B.  WARRANTY  IMPACTS  UNDER  SPECIFICATION  RELIEF 

1 . The  Relationship  Between  Specification  Relief  and 

Warranty  Impact 

Increased  specification  relief  was  reported  to  be  a poten- 
tial condition  under  which  the  impact  of  warranties  upon  the 
design  and  development  process  would  be  increased.  The  situation 
to  be  considered  is  one  in  which  all  specifications  and  require- 
ments are  flexible  and  negotiable  except  for  critical  performance, 
cost,  and  reliability  requirements. 

The  possibility  of  granting  specification  relief  stems  from 
the  perceived  responsibility  of  the  contractor,  under  a warranty, 
to  maintain  his  equipment  over  an  extended  period  of  time. 
Specification  relief  would  allow  contractors  to  transfer  their 
efforts  from  meeting  certain  specifications  to  more  productive 
areas  by  allowing  them  freedom  to  design,  test,  produce,  and 
maintain  equipment  in  a manner  consonant  with  contractor 
maintenance . 

The  elimination  of  unnecessarily  restrictive  specifica- 
tions has  been  advocated  by  many  observers.  The  Electronics-X 
study,  for  example  reported,  "though  these  reference  specifica- 
tions are  frequently  related  only  remotely  and  indirectly  to 
the  objective  of  realizing  the  desired  performance  of  the  elec- 
tronic system  to  which  they  apply,  they  provide  a ready  basis 
for  rejection  by  Government  inspectors  of  equipment  that  is 
totally  satisfactory  from  a functional  standpoint."  Electronics- 
X also  noted  the  successful  approach  of  the  commercial  air 
transport  industry,  from  which  the  warranty  concept  came,  which 
allows  much  more  freedom  to  manufacturers.  One  factor  cited  as 
leading  to  the  success  of  this  approach  was  summarized  this 
way:  "the  widespread  provision  of  long-term  warranties  by 

equipment  suppliers  assures  the  purchaser  that  he  will  get  the 
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required  functional  performance  despite  his  not  specifying 
internal  design  details."1 

The  effects  of  increased  specification  relief  are  believed 
to  be  twofold — first,  contractors  can  be  innovative  in  their 
designs  and  program  planning;  and  second,  contractors  can  trans- 
fer time,  money,  and  manpower  required  as  part  of  the  obliga- 
tions associated  with  military  specifications  to  areas  of  effort 
more  compatible  with  warranty  features. 

Some  military  specifications  affect  the  impact  of  warran- 
ties more  than  others.  These  would  include,  for  example, 
reliability  and  maintainability  demonstration  test  requirements. 
If  these  specifications  were  altered  or  waived,  the  contractors 
could  afford  to  design  and  implement  a series  of  reliability 
tests  geared  more  to  field  reliability  predictions.  The  argu- 
ment for  specification  relief  has  been  made  frequently,  but 
without  a warranty  the  government  heretofore  has  net  had  any 
assurance  of  product  quality  under  specification  relief.2 

2 . Contractor  Responses 

The  subsystem  contractors  were  asked  to  estimate  the  impact 
on  the  fifty-two  program  elements  listed  in  the  questionnaire 
under  the  condition  that  the  warranty  requirement  was  firm  and 
in  addition  all  specifications  and  requirements  were  flexible 
and  negotiable  except  for  critical  performance,  cost,  and 
reliability  requirements.  The  questionnaire  results  are  dis- 
played in  Table  16.  The  responses  were  then  compared,  as  before, 
with  the  impacts  estimated  for  the  firm  and  actual  cases,  as 
shown  in  Table  17.  Table  17  reveals  that  a general  reduction 


1 Howard  P.  Gates,  Jr.,  et  al.,  Electronica-X:  A Study  of  Military  Elec- 

tronics with  Particular  Reference  to  Cost  and  Reliability , in  2 vols., 
IDA  Report  R-195  (1974),  II:  290. 

2Except  for  the  standard  correction-of-deficiencies  contract  clause. 
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Table  16. 


POTENTIAL  WARRANTY  IMPACT  UNDER  SPECIFICATION  RELIEF 


1 AVERAGE  (PERCENT)*** {g.gp  19.62  3.6S  0.96  0.S8  0.96  0 16.00  4.23  B.CB  8.23  0 -2.86  J 

♦Contractor  P did  not  respond  to  this  condition;  therefore  averages  are  based  on  15  contractors. 

•♦Italicized  numbers  for  each  major  heading  are  totals  of  subelements. 

***The  average  impact  Is  based  on  the  average  of  all  subelements.  Because  of  rounding,  the  average  of  the  major  element 
totals  Is  5.59  percent. 
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in  effort  beyond  that  reported  for  the  firm  case  would  be  the 
result  under  conditions  of  specification  relief.  Table  18  sum- 
marizes the  reduction  of  effort  for  each  of  the  development 
program  groups  from  the  total  contractor  sample. 

A survey  of  the  sixteen  contractor  development  programs 
reveals  that  four  showed  no  change  due  to  specification  relief, 
three  showed  an  increase  in  effort  due  to  specification  relief, 
and  the  remaining  seven  showed  reductions  in  effort  ranging  up 
to  30  percent. 

Table  18.  REDUCTION  IN  EFFORT  DUE  TO  SPECIFICATION  RELIEF 


Percent  Reduction 

Data  and  Reports 

9.33 

Quality  Assurance 

7.50 

Program  Management 

7.33 

Equipment  Test 

6.78 

Materials/Purchased  Parts 

6.45 

Prototype  Manufacturing 

5.87 

Design 

5.33 

Analysis 

3.86 

Product  Support 

3.84 

Production  Engineering 

3.33 

Financial  Management 

3.17 

Contract  Management 

1.33 

Value  Engineering 


0 

3.  Specifications  as  Impact  Barriers 

Many  of  the  Government  requirements  and  specifications 
that  are  typically  imposed  on  a design  and  development  program 
were  observed,  as  noted  in  Chapter  IV,  to  be  barriers  to  effec- 
tive warranty  impact.  The  questionnaire  responses  indicated 
several  major  requirements  and  specifications  that  contractors 
would  like  to  have  altered  or  waived.  The  following  were  most 
frequently  mentioned: 

• MIL-E-5^ OOP , General  Specification  for  Airborne 
Electronic  Equipment 
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• MIL-STD-781,  Reliability  Tests 

• MIL-STD-785,  Reliability  Programs 

• MIL-STD-810,  Environment  Qualification  Tests 

• MIL-STD-454 , General  Requirements  for  Electronic 

Equipment 

• MIL-STD-883,  Test  Methods  and  Procedures  for 
Microelectronics 

• MIL-M-38510,  General  Specifications  for  Microcircuits 

• MIL-STD-275,  Criteria  on  Printed  Circuit  Construction 

• MIL-STD-704,  Aircraft  Electric  Power  Characteristics 

• MIL-Q-9858,  Quality  Assurance  and  Engineering. 

Contractors  also  listed  generic  classes  of  requirements 
and  specifications  that  should  be  changed. 


• Configuration  management  specifications 

• GSE  specifications 

• Quality  assurance  and  control  specifications 

• Reliability  demonstration  specifications 

• Maintainability  demonstration  specifications 

• Methods  of  collecting  and  evaluating  field  malfunction 
reports 

• Parts  selection  criteria  and  standard  parts  lists 

• ECP  control 

• Non-standard  parts  approval 

• Reprocurement  data  packages. 

The  basic  rationale  offered  by  the  contractors  for  seeking 
relief  or  elimination  of  some  of  the  above  specifications  bar- 
riers is  that  if  a contractor  is  to  be  ultimately  responsible, 
through  a warranty,  for  the  maintenance  of  his  equipment,  then 
he  should  be  allowed  the  freedom  to  design,  test,  and  produce 
equipment  the  way  he  wants.  Contractors  said  that  the  perform- 
ance guarantee  of  a warranty  negates  the  need  for  many  of  the 
thousands  of  applicable  specifications  by  assuring  the  purchaser 
that  he  will  get  the  required  functional  performance  without 
detailed  specification  of  internal  design  details  or  development 
processes . 


85 


c. 


CORRELATION  ANALYSIS  OF  CONTRACTOR  RESPONSES 


Throughout  the  study  analyses,  an  attempt  was  made  to  deter- 
mine if  a relationship  existed  between  contractor  response  and 
program  status.  Service  sponsor,  competitive  environment,  or 
other  contractor  or  program  characteristics. 

Table  19  below  summarizes  the  data  used  for  these  analyses. 
The  aggregate  impact  for  all  program  elements  is  presented  to- 
gether with  two  complementary  subsets  consisting  of  those 
program  elements  representing  activities  directly  related  to 
development  program  conduct  and  those  of  indirect  influence. 

The  data  suggest  that  contractors  also  weighted  these  two 
categories  differently. 

Tables  20  and  21  illustrate  some  of  the  more  important 
contractor  subsets  that  were  examined  for  correlation  to  or 
departure  from  the  aggregate  and  direct  program  element  impacts. 
The  F-16  contractors,  for  example,  reported  an  average  impact 
similar  to  the  total  sample,  except  for  the  actual  case,  in 
which  the  F-16  contractors  were  essentially  operating  under 
beginning  conditions.  The  advanced  "3,"  consisting  of  those 
contractors  associated  with  the  three  programs  in  the  production 
stage  (ARN-118  Tacan,  ARC-164  radio,  APN-209  altimeter),  re- 
ported a higher  average  impact  for  each  condition  than  did  the 
total  sample.  However,  a statistical  comparison  of  the  sample 
means  and  variances  did  not  indicate  that  the  impact  reported 
by  this  group  was  significantly  different  from  that  reported  by 
the  others.  Other  distinctions  that  are  apparent  in  Tables  20 
and  21  include  the  higher  average  impact  reported  by  Air  Force 
program  contractors  and  the  similarities  in  program  impact 

between  the  averages  of  all  program  elements  and  the  average 

« 

of  the  direct  program  elements. 

Tables  20  and  21  also  show  that  differences  exist  among 
the  average  subset  impacts  under  each  condition,  but  that  when 
the  standard  deviation  is  calculated  and  statistical  analysis 
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21.  SUBSET  CORRELATION  ANALYSIS:  DIRECT  IMPACT  PROGRAM  ELEMENTS 


is  attempted,  these  differences  are  not  statistically  signifi- 
cant. The  subset  impacts  are  characterized  more  by  their 
similarities  than  by  their  differences. 


Attempts  to  discern  additional  significant  groupings  based 
on  reported  impacts  were  unsuccessful.  It  is  postulated  that 
other  factors  are  responsible  for  portions  of  the  contractor 
responses  and  that  those  factors  are  either  random  in  applica- 
tion or  more  subtle  than  those  explicitly  addressed.1 

D.  IMPACT  MOTIVATION  AND  CHANGE  IMPLEMENTATION 

The  research  findings  thus  far  have  demonstrated  that 
changes  are  occurring  during  design  and  development  as  a result 
of  warranty  considerations.  At  the  same  time,  there  are  bar- 
riers that  prevent  greater  impacts  from  occurring.  If  some  of 
those  barriers  were  removed,  a significant  change  in  develop- 
ment program  effort  would  possibly  occur. 

This  section  identifies  some  of  the  motivations  behind  the 
potential  impacts  that  have  been  identified  and  the  steps  con- 
tractors believe  could  be  taken  to  implement  changes  in  the 
typical  electronics  subsystem  design  and  development  program. 

1 . Risk  and  Uncertainty:  The  Primary  Motivators 

There  is  no  question  that  the  primary  motivation  for  con- 
tractors to  implement  changes  during  design  and  development  is 
to  reduce  future  product  experience  risk.  Information  is 
sought  that  will  reduce  risk  in  the  following  areas: 

• Initial  field  reliability 

• Field  reliability  growth 

• Probable  failure  modes 

• Operational  environment 

I 

lrthe  reader  is  directed  to  the  discussion  in  Chapter  VII  for  the  identifi 
cation  of  additional  factors  that  were  found  to  influence  warranty  appli 
cation  and  impact. 
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• Unit-production  costs 

• Economic  escalation  effects 

• Possibility  of  induced  failures 

• Expected  logistics  system  experiences 


The  most  frequently  mentioned  area  of  risk  was  operational 
reliability  prediction  for  production  units.  This  area  and  the 
risks  associated  with  pricing  replacement  units  or  consignment 
spares  years  into  the  future  combine  to  yield  large  financial 
risks  that  cannot  entirely  be  quantified  through  development 
actions . 


Reported  Mays  to  Reduce  Risks 


Several  methods  were  suggested  by  the  contractors  to  help 
reduce  risks.  It  has  already  been  found  that  changes  in  warranty 
decisionmaking  and  the  removal  of  a number  of  barriers  would  per- 
mit contractors  to  change  their  development  program  efforts  to 
better  quantify  their  risks.  Another  commonly  reported  approach 
is  to  expand  the  development  pilot  production  phase  (or  low-rate 
initial  production)  such  that  some  operational  reliability  and 
performance  data  can  be  obtained  prior  to  warranty  pricing. 

Listed  below  are  other  proposed  actions  that  were  recommended. 


Defer  warranty  price  commitment  until  adequate 
technical  and  performance  data  exist. 

Develop  procedures  whereby  Government  can  budget  for 
and  negotiate  multi-year  contracts. 

Specify  performance  rather  than  design  parameters  to 
suppliers . 

Do  not  price  or  negotiate  during  competitive  phase. 

Extend  the  development  phase  to  allow  sufficient 
time  for  consideration  of  design  options:  structure 

an  intensive  post-development  test  program,  incor- 
porating field  tests,  to  give  the  contractor  and 
Government  assurance  and  data  prior  to  entering  an 
RIW  contract. 

Delete  induced  failures  from  contractor  responsibility 
under  warranty  conditions. 


(7)  Define  specific  aircraft  environments  more  precisely. 

(8)  Establish  pilot  production  runs. 

(9)  Allow  more  performance-reliability  trade-offs. 

(10)  Price  options  after  prototype  flight  testing. 

(11)  Allow  for  renegotiation  on  first  warranties  after 
field  experience  data  have  been  gathered  and 
evaluated. 

(12)  Define  specific  environmental  and  operational 
applications. 

(13)  Conduct  additional  cost  analyses  of  repair  costs 
with  varying  failure  modes. 

The  above  list  of  contractor  recommendations  was  augmented 
by  questionnaire  responses  to  several  questions  that  focused  on 
specific  areas  of  effort  needed  to  reduce  risk.  The  most  com- 
mon response  was  a desire  for  longer  and  more  realistic  test 
programs.  Examples  of  areas  in  which  additional  testing  was 
recommended  were — 

(1)  Flight  testing. 

(2)  Formal  reliability  improvement-corrective  action 
program. 

(3)  Operational-type  testing  in  development. 

(M)  Accelerated-life  testing. 

(5)  Mock-up  testing. 

(6)  More  burn-in  type  testing. 

(7)  Scheduled  tests  from  initial  production. 

(8)  More  realistic  reliability  testing. 

Based  on  the  contractor  responses,  the  greatest  change  would  be 
implemented  in  the  area  of  reliability  demonstration  testing 
to  reduce,  as  much  as  possible,  the  uncertainties  associated 
with  predicting  future  field  reliability.  Changes  would  also 
be  implemented  in  prototype  or  pilot  production  programs  to 
provide  test  data  on  equipment  that  most  closely  resembles  the 
production  configuration  and  quality. 
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The  potential  impact  of  warranties  on  the  candidate  develop 
ment  programs  was  investigated  and  quantified  under  three  postu- 
lated conditions  of  warranty  application.  In  addition,  the 
factors  influencing  the  impacts  were  evaluated,  potential  areas 
of  program  change  were  identified,  and  correlation  of  individual 
responses  with  contractor  and  program  characteristics  was 
attempted. 


Potential  Development  Program  Impact  When  Warrant 
Option  Terms  and  Conditions  are  Known  at  the 
Beginning  of  Engineering  Development 


It  was  found  that,  for  the  aggregate  contractor  sample, 
the  total  average  development  program  impact  would  represent 
an  8.2  percent  increase  in  effort  if  warranty  option  terms  and 
conditions  were  known  at  the  beginning  of  engineering  develop- 
ment. This  represented  a 50  percent  increase  in  impact  over 
that  currently  attributed  to  the  prospect  of  warranties  in  the 
candidate  programs.  The  major  absolute  increases  in  effort 
were  predicted  in  areas  of  data  and  reports,  materials/purchased 
parts,  analysis,  equipment  test,  and  design.  The  areas  of 
greatest  impact  change  (leverage)  were  found  to  be  value  engi- 
neering, prototype  manufacture,  production  engineering,  and 
equipment  test.  If  those  program  elements  receiving  an  impact 
other  than  zero  were  considered,  there  was  no  appreciable 
change  in  impact  on  development  effort  over  the  actual  candi- 
date cases. 


Potential  Development  Program  Impact  When  Warrant 
is  a Firm  Requirement 


It  was  found  that  a firm  warranty  requirement  would  result 
in  an  absolute  increase  in  effort  of  10.5  percent,  which  repre- 
sented a 90  percent  increase  over  the  impacts  presently  experi- 
enced, on  the  average.  Major  functional  areas  affected  were 


prototype  manufacturing,  production  engineering,  design,  and 
quality  assurance.  Program  management  was  the  area  receiving 
the  highest  absolute  increase  in  effort — 14  percent.  If  only 
those  program  elements  receiving  an  impact  other  than  zero 
were  analyzed,  an  absolute  15  percent  increase  in  effort  was 
estimated,  which  represented  a 22  percent  increase  in  effort 
over  the  present  programs.  Highest  areas  of  impact  included 
data  and  reports,  program  management,  production  engineering, 
and  materials/purchased  parts. 

3.  Potential  Development  Proqram  Impact  Under  Specification 

Rel ief 

It  was  found  that  if  non-critical  specifications  and 
standards  were  relaxed,  contractors  believed  that  the  develop- 
ment program  efforts  could  be  accomplished  more  efficiently 
than  in  the  previous  two  cases,  such  that  an  average  nominal 
5.6  percent  increase  would  be  experienced.  This  is  essentially 
the  same  change  in  effort  as  presently  experienced.  If  only 
those  program  elements  receiving  an  impact  other  than  zero  are 
considered,  the  decrease  in  effort  is  even  greater,  with  the 
average  percentage  increase  in  effort  declining  from  12.4  per- 
cent to  9-9  percent.  Data  and  reports,  quality  assurance, 
program  management,  and  equipment  test  were  the  key  areas  in 
which  decreases  in  effort  were  projected. 


4 . Correlation  of  Contractor  Responses 

It  was  found  that  certain  contractors  consistently  reported 
greater  increases  in  development  program  effort  than  others  over 
all  conditions  examined.  For  those  contractors  reporting  a 
warranty  impact  other  than  zero,  the  estimated  impact  on  the 
candidate  development  program  averaged  12.4  percent.  The 
reported  impacts  for  the  beginning,  firm,  and  specification 
relief  conditions  were  12.8,  15.1,  and  9-9  percent,  respectively. 
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Except  for  the  actual  case,  this  contractor  subset  reported  an 
average  impact  60  percent  higher  than  the  total  aggregate 
sample.  Attempts  to  correlate  this  group  of  contractors  with 
other  contractor  characteristics  were  unsuccessful. 


Impac t Motiva t i o n a n dCh  a n < 
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It  was  found  that  the  greatest  motivation  for  changes  in 
the  engineering  development  program  was  the  desire  to  reduce 
financial  risk  and  uncertainty  associated  with  field  reliability 
prediction  and  future  product  cost.  The  changes  driven  by 
these  motivations  were  reported  to  be  widespread;  more  extensive 
test  programs  (not  necessarily  more  expensive)  and  expansion  of 
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THE  IMPACT  OF  WARRANTIES  ON  CANDIDATE  EQUIPMENT  DESIGNS 

The  improvement  of  equipment  reliability  through  changes 
in  design  is  a key  objective  of  warranties.  This  chapter  exam- 
ines some  of  the  changes  in  design  that  occurred  as  a result  of 
warranty  considerations  and  the  requirements  that  drove  those 
changes.  The  previous  chapters  reported  an  average  development 
program  impact  of  from  5-9  percent  increase  in  design  effort; 
accordingly,  corresponding  impacts  were  found  in  the  candidate 
product  designs. 

This  chapter  focuses  on  the  product  of  the  design  effort — 
a black  box  having  certain  design  features,  partly  as  a result 
of  the  warranty  requirements.  Based  on  contractor  interviews 
and  the  responses  to  the  research  questionnaire,  specific 
design  impacts  have  been  categorized  into  five  areas  for 
analysis : 

• Contractually  required  items 

• BITE-automatic  test  equipment 

• System  architecture 

• Sensors,  indicators,  and  protective  circuits 

• Design  redundancy  and  parts  selection  for  higher  in- 
herent reliability. 

The  major  design  impacts  are  summarized  in  Table  22. 

A.  CONTRACTUALLY  REQUIRED  DESIGN  IMPACTS 

Our  research  found  several  warranty  provisions  that  led 
directly  to  design  impacts  by  requiring  the  inclusion  of  certain 
features  in  the  products.  Tamper-proof  seals,  for  example,  had 
to  be  Incorporated  in  equipment  box  designs  so  that  unauthorized. 

97 


- 


! 


entry  or  repair  could  be  easily  detected.  The  use  of  a seal  is 
intended  to  reduce  tampering,  which  can  range  from  attempts  at 
minor  repairs  to  extensive  cannibalization. 


Warranty  markings  were  also  required  to  advise  users  that 
an  equipment  warranty  is  in  effect,  thereby  warning  the  user 
not  to  void  the  warranty  by  attempting  unauthorized  repairs. 
Labeling  units  to  allow  the  logging  of  information,  such  as 
installation-removal  dates,  aircraft  flying  hours,  and  serial 
number,  has  also  been  required  under  warranty  clauses. 

Elapsed-time  indicators  (ETI)  were  usually  required  by 
contract  to  verify  the  actual  equipment  operating  hours.  This 
requirement  is  critical  because  the  empirical  parameter,  equip- 
ment operating  hours,  enters  into  several  key  cost-sensitive 
provisions  of  warranty  clauses. 

B.  BITE-AUTOMATIC  TEST  EQUIPMENT 

The  major  responsibility  of  a contractor  under  a warranty 
is  to  repair  all  faulty  equipment  returned  by  the  customer. 

Since  the  contractor  is  responsible  for  the  repair  costs,  he 
should  be  motivated  to  take  steps  in  the  design  of  his  equipment 
to  reduce  those  expenses.  One  approach  is  to  "design  in"  higher 
inherent  reliability  so  that  the  calculated  equipment  MTBF  is 
sufficiently  high  (see  Section  E) . 1 Another  approach  is  to 
provide  for  determining  the  failure  mode  of  a failed  unit  even 
before  it  arrives  at  the  plant  so  that  appropriate  spares  can 
be  available  and  other  maintenance  preparations  taken.  These 
actions  would  reduce  the  MTTR  and  associated  cost-to-repair . 

A related  item  of  expense  to  contractors  is  RTOK  items — 
returned  equipment  that  subsequently  is  found  to  be  working  cor- 
rectly. The  checkout  process  for  this  equipment  can  be 

1 "Designing  in"  reliability  most  often  consists  of  selecting  component  parts 
with  higher  reliability  ratings  or  designing  circuits  with  redundant,  fail- 
safe features. 
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expensive — significantly  higher  than  the  $100  reimbursement  for 
RTOKs  cited  in  some  warranty  contracts.  It  is  therefore  to  the 
contractor's  advantage  to  reduce  the  number  of  RTOKs. 

The  observed  design  impact  in  this  area  is  an  increased 
emphasis  on  built-in  test  equipment  and  automatic  test  equipment. 
These  impacts  can  range  from  including  such  items  as  go/no-go 
testers  or  externally  viewed  failure  flags,  to  designing  exten- 
sive automated  test  equipment,  either  for  field  or  contractor 
use.  The  basic  purpose  of  the  test  equipment  is  to  aid  in  fault 
verification,  isolation,  and  diagnosis,  and  to  assist  in  inter- 
mittent failure  detection  and  verification,  thereby  reducing 
MTTR  and  RTOKs. 

The  impact  on  test  equipment  design  caused  by  requirements 
for  fault  isolation  and  diagnosis  is  an  important  consideration 
for  contractors,  since  most  equipments,  even  if  meeting  the 
specified  MTBF,  will  fail  once  or  twice  during  the  time  frame 
of  a multi-year  RIW  contract  and  will  be  returned  to  the  con- 
tractor for  repair.  Contractors  noted  in  our  interviews  that 
the  design  for  BITE  is  closely  related  to  the  maintenance  level 
(e.g.,  LRU,  SRA , module)  for  which  it  is  intended. 

C.  SYSTEM  ARCHITECTURE 

The  system  architecture  element  of  the  design  process  was 
also  found  to  be  influenced  by  the  maintenance  level  for  which 
the  equipment  is  designed.  Many  design  features  can  be  employed 
to  enhance  contractor  repair  and  thereby  significantly  lower 
MTTR  and  the  corresponding  cost-of-repair . 

The  circuit  partitioning  among  boards,  cards,  and  modules 
can  be  arranged  in  a manner  to  better  suit  the  use  of  the  con- 
tractor's factory-based  test  equipment.  At  a higher  maintenance 
level,  the  design  of  a set  of  autonomous  LRUs  negates  the  re- 
quirement to  ship  an  entire  failed  unit  back  to  the  factory.1 


'in  many  conplex  systems,  each  LRU  must  be  "triimed"  or  specifically  tuned 
for  the  rest  of  the  system  and  therefore  is  not  an  autonomous  unit. 
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The  accessibility  of  LRUs  and  SRUs  is  a design  feature  also 
receiving  increased  attention. 


Warranty  considerations  were  found  to  affect  contractor 
make-buy  decisions  in  the  area  of  maintaining  spares  levels. 

In  one  candidate  equipment,  for  example,  alignment  was  critical 
to  meeting  performance  requirements.  The  contractor  could 
either  purchase  the  complete  assembly  (already  aligned)  or 
purchase  the  parts  and  assemble  and  then  align  them.  The 
consideration  of  having  to  furnish  an  expensive  spare  (the 
aligned  subassembly  purchased  from  vendors)  convinced  the  con- 
tractor to  assemble  this  component  himself  to  reduce  his 
potential  spares  cost  under  the  warranty,  even  though  the  pur- 
chased aligned  assembly  might  be  preferable  in  performance. 


D.  SENSORS,  INDICATORS,  AND  PROTECTIVE  CIRCUITS 

Contractors  can  employ  a series  of  design  strategies  to 
monitor  the  operating  environment  of  their  equipment  in  the 
field.  Of  particular  concern  is  the  thermal  environment,  but 
the  correct  application  of  primary  electrical  power,  vibration 
and  shock  conditions,  and  radiation  hazards  also  have  to  be 
considered. 


In  some  cases,  the  contractor  can  obtain  an  "induced  fail- 
ure" exclusion  from  the  warranty  if  the  operational  environment 
experienced  by  the  equipment  can  be  shown  to  be  outside  speci- 
fications. In  such  cases,  an  indicator  of  thermal  overload  or 
an  over-under  voltage  Indicator  can  help  assess  responsibility 
for  the  equipment  failure . 

In  other  cases  we  examined,  the  lack  of  proper  environmen- 
tal conditioning  is  not  necessarily  cause  for  a failure  exclusion. 
In  these  cases,  the  contractor  provided  sensors  linked  to  pro- 
tective circuitry  in  order  to  protect  his  equipment.  For 
example,  a monitoring  sensor  can  be  designed  to  shut  off  the 
equipment  if  there  is  a deficiency  in  the  cooling  air  input  to 
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the  equipment.  Upon  a post-operation  inspection,  a flag  can 
then  be  read  to  determine  that  this  condition  obtained.  Even 
if  protective  circuitry  is  not  used,  indicators  of  "out-of-spec" 
conditions  are  helpful  in  failure  diagnosis.  Other  design  fea- 
tures in  use  to  prevent  potential  environmental  conditioning 
problems  included  the  use  of  low-power  electronics  and 
conduction-cooled  electronics  to  reduce  dependence  on  cooling 
air. 


E.  DESIGN  FOR  HIGHER  INHERENT  RELIABILITY 

We  expected  that  the  impact  of  warranties  on  the  design 
process  would  include  a number  of  design  changes  to  improve  the 
inherent  reliability  of  the  equipment  under  development.  Our 
research  found  that  this  was  not  an  important  impact.  Pew 
contractors  reported  the  use  of  component  or  circuit  redundancy1 — 
a classic  technique  for  improving  reliability. 

The  only  impact  that  was  reported  was  a greater  tendency 
to  use  proven  integrated  circuits  (ICs).  Vendors  were  being 
asked  to  show  examples  of  existing  successful  applications  in 
addition  to  the  usual  "brochuresmanship"  estimates  of  reliabil- 
ity. Several  contractors  also  cited  derating  of  parts  and 
components  as  becoming  more  common.2 

F.  FINDINGS 

Changes  in  product  design  as  a result  of  the  warranty  re- 
quirements were  reported  by  the  contractors.  Warranty  require- 
ments led  to  design  changes  in  areas  of  tamper-proof  seals, 
warranty  markings  and  labeling,  and  elapsed-time  indicators. 


!It  is  recognized  that  the  existence  of  DTC  goals  for  unit-production  costs 
tends  to  inhibit  such  design  actions. 

2 Derating  involves  using  a part  at  an  electrical-environmental  operating 
level  less  than  the  maximum  for  which  it  was  designed.  For  example,  a 
part  may  be  rated  at  a certain  maximum  tenperature  and  derating  would  mean 
using  that  part  only  in  applications  of  25  percent  lower  teirperatures . 
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More  sophisticated  designs  for  BITE  and  automated  test  equipment 
were  used  by  contractors  to  reduce  the  MTTR  and  number  of  RTOKs . 
The  system  architecture  was  also  changed  to  enhance  contractor 
repair.  Sensors,  indicators,  and  protective  circuits  were  be- 
coming important  design  ingredients  in  order  to  monitor  the 
equipment  operating  and  handling  environment.  Finally,  the  use 
of  proven  components  and  derating  techniques  to  improve  inherent 
reliability  was  also  reported. 
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VII 

ADDITIONAL  ISSUES  RELATING  TO  WARRANTY  IMPACTS 

The  research  efforts  expended  during  this  study,  and 
particularly  the  numerous  meetings  with  both  Government  program 
officers  and  contractor  personnel,  uncovered  a number  of  addi- 
tional issues  and  problem  areas  that  can  also  affect  the  appli- 
cation and  impact  of  reliability  warranties.  Many  are  linked 
to  other  problems  needing  solution.  Some  can  be  trivial  in 
major  programs  or  system  developments,  but  can  become  major 
roadblocks  at  the  subsystem  level.  Some  require  political  or 
institutional  solutions.  All  need  more  investigation  and  more 
illumination. 

A.  CONTRACTOR  MOTIVATION  FOR  INCREASED  RELIABILITY 

Many  of  the  contractors  and  Government  personnel  we  inter- 
viewed believe  that  a major  failing  of  the  present  electronics 
system  and  subsystem  acquisition  process  is  that  it  permits  the 
contractor  to  proceed  into  production  without  sufficient  con- 
fidence or  assurance  that  predicted  field  performance  will  be 
achieved.  Current  proof-test  programs  are  not  considered 
adequate;  often  the  primary  motivation  for  contractors  is  to 
"pass  the  test"  rather  than  to  test  and  fix  equipment  rigorously 
and  repeatedly  until  it  has  been  thoroughly  evaluated.1 

The  prospective  exercise  of  warranty  options  by  the 
Government  was  expected  to  change  this  "pass  the  test"  attitude. 


*It  is  not  so  much  a case  of  poor  test  design  as  it  is  of  inadequate  relevant 
failure  interpretation,  non-realistic  test  environment,  use  of  handmade 
prototype  test  equipment,  and  skilled  contractor  maintenance  personnel. 

In  many  development  and  demonstration  programs,  the  pressures  to  announce 
the  successful  completion  of  the  equipment  development  often  take  precedence 
over  a realistic  assessment  of  program  status. 
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But  our  research  found  little  change  in  traditional  contractor 
motivation.  Many  of  those  interviewed  believe  that  unless  the 
penalties  (not  necessarily  cost)  outweigh  the  advantages  of 
"passing  the  test"  and  unless  steps  are  taken  to  provide  the 
time  and  funds  necessary  to  develop  complex  equipment  properly, 
there  will  be  no  real  contractor  motivation  for  ensuring  that 
design  reliability  is  transferred  through  development  to  field 
operations . 1 

B.  GOVERNMENT  CREDIBILITY  FOR  WARRANTY  SUCCESS 

Throughout  our  investigations  on  warranty  impact,  the  ques- 
tion of  Government  credibility  was  frequently  raised.  In  large 
measure,  we  found  this  question  to  be  founded  on  the  contractor's 
real  or  perceived  observation  that  project  officers  were  merely 
going  through  policy-directed  motions  without  any  real  interest 
in  negotiating  and  implementing  warranties.  Frequently,  warranty 
options  were  observed  to  be  unpopular  with  Government  program 
officers  because  the  details  of  the  warranty  were  not  fully  under- 
stood, were  difficult  to  negotiate,  were  almost  impossible  to 
substantiate  on  the  basis  of  cost  alone,  were  a financial  burden 
on  scarce  procurement  funds,  and  would  ultimately  be  supervised 
and  directed  by  another  functional  organization  within  the 
Services . 

In  addition  to  perceiving  Government  hesitancy  over  war- 
ranties, some  contractors  believe  that,  if  the  Government  is 
truly  serious,  they  should  be  allowed  more  freedom  to  design 
and  develop  equipment  in  a way  that  minimizes  their  warranty 


1Somehow,  almost  all  newly  developed  equipment  manages  to  "pass  the  test" 
during  reliability  demonstration,  acceptance,  and  other  performance 
verification  or  validation  tests.  Test  waivers,  definitions  of  relevant 
failures,  contingency  provisions,  and  similar  test  modifiers  are  all  used 
to  demonstrate  the  successful  test  accomplishment , usually  on  prototype 
hardware — which  may  differ  substantially  frcm  production  hardware. 
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risks.  Unfortunately,  as  discussed  in  Chapter  IV,  in  an 
environment  of  tight  budgets  and  constrained  program  schedules, 
such  freedom  is  not  easily  achieved. 

Thus,  many  contractors  feel  that  it  is  "business  as  usual" 
and  have  discounted  the  possibility  of  having  to  stand  behind  a 
product  warranty  that  might  ultimately  result  in  great  financial 
loss . 1 

C.  COMPETITIVE  ENVIRONMENT 

As  in  all  programs,  the  competitive  environment  influences 
the  actions  of  the  individual  contractor  firm  throughout  the 
equipment  acquisition  phase.  During  our  research,  we  found  many 
instances  in  which  competitive  efforts  to  capture  the  market, 
perpetuate  the  product  line,  or  acquire  new  technology  influenced 
decisions  during  engineering  development.2  Too  often,  warranty 
options  were  priced  so  as  to  remain  competitive,  without  a full 
appreciation  of  later  development  and  production  impacts.  Simi- 
larly, development  programs  were  often  planned  and  proposed  to 
meet  customer  demands  for  schedule  compression  and  low  cost. 

In  this  environment,  the  impact  of  future  warranty  obligations 
is  latent  until  a full  appreciation  of  future  obligations  and 
financial  risk  is  acquired.  Under  these  circumstances,  the 
relationship  of  equipment  performance  and  contract  requirements 
to  design  and  development  program  planning  as  well  as  the 
relationship  of  price  to  actual  estimated  costs,  including 


1Many  contractors  were  surprised  that  the  first  major  tjarranty  contract  (Air 
Force  Tacan)  was  actually  executed  with  the  RlW-guaranteed  MIEF  provisions. 
Some  still  believe  that  sufficient  legal  remedies  exist  to  preclude  major 
losses  on  the  Tacan  or  similar  programs. 

2rIhis  is  not  to  be  construed  as  a criticism  of  ccnpetition,  which  probably 
leads  to  more  efficient  management.  However,  the  power  of  a monopsonistic 
market  can  force  sellers  to  take  otherwise  unacceptable  risks  to  remain  in 
the  market. 
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risks.  Unfortunately,  as  discussed  in  Chapter  IV,  in  an 
environment  of  tight  budgets  and  constrained  program  schedules, 
such  freedom  is  not  easily  achieved. 

Thus,  many  contractors  feel  that  it  is  "business  as  usual" 
and  have  discounted  the  possibility  of  having  to  stand  behind  a 
product  warranty  that  might  ultimately  result  in  great  financial 
loss . 1 

C.  COMPETITIVE  ENVIRONMENT 

As  in  all  programs,  the  competitive  environment  influences 
the  actions  of  the  individual  contractor  firm  throughout  the 
equipment  acquisition  phase.  During  our  research,  we  found  many 
instances  In  which  competitive  efforts  to  capture  the  market, 
perpetuate  the  product  line,  or  acquire  new  technology  influenced 
decisions  during  engineering  development.2  Too  often,  warranty 
options  were  priced  so  as  to  remain  competitive,  without  a full 
appreciation  of  later  development  and  production  Impacts.  Simi- 
larly, development  programs  were  often  planned  and  proposed  to 
meet  customer  demands  for  schedule  compression  and  low  cost. 

In  this  environment,  the  impact  of  future  warranty  obligations 
is  latent  until  a full  appreciation  of  future  obligations  and 
financial  risk  is  acquired.  Under  these  circumstances,  the 
relationship  of  equipment  performance  and  contract  requirements 
to  design  and  development  program  planning  as  well  as  the 
relationship  of  price  to  actual  estimated  costs,  including 


1Many  contractors  were  surprised  that  the  first  major  warranty  contract  (Air 
Force  Tacan)  was  actually  executed  with  the  RlW-guaranteed  MIBF  provisions. 
Some  still  believe  that  sufficient  legal  remedies  exist  to  preclude  major 
losses  on  the  Tacan  or  similar  prop  'ms. 

2This  is  not  to  be  construed  as  a criticism  of  competition,  which  probably 
leads  to  more  efficient  management.  However,  the  power  of  a monopsonistic 
market  can  force  sellers  to  take  otherwise  unacceptable  risks  to  remain  in 
the  market. 
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risk  coverage,  can  be  heavily  biased  in  order  to  respond  to 
market  forces.1 

D.  PREDICTION  OF  FIELD  RELIABILITY 

Prediction  of  field  reliability  has  been  mentioned  through- 
out this  study  as  a source  of  negotiating  difficulty,  a consid- 
eration that  can  impact  on  the  equipment  design  and  development 
program,  and  as  the  major  uncertainty  in  warranty  pricing  or 
risk  assessment.  This  issue  has  also  had  a peripheral  impact  on 
development  programs  by  establishing  new  data  requirements  for 
the  benefit  of  future  warranty  programs. 

As  the  need  for  an  accurate  estimate  of  a new  product's 
field  reliability,  both  initially  and  as  a function  of  operating 
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time,  became  critical,  the  need  for  better  field  data  on  the 
reliability  experience  of  past  systeks  was  also  recognized. 
Subsystem  contractors  discovered  that  they  had^poor  or  incomplete 
data  bases  for  their  own  equipment  and  also  for'^ther  similar 
equipment  presently  in  the  field.2  Moreover,  mucN  of  their 
data  was  of  questionable  utility  since  information Vn  important 
variables,  such  as  operating  time,  application,  and 'cause  of 
failure,  were  not  readily  available  from  Government  maintenance 
reports.3  We  found  that  this  void  in  the  basic  information 
required  to  determine  historical  relationships  between  demon- 
strated and  operational  reliability  has  created  a need  to  develop 
a permanent  data  base  on  past  and  present  equipment  reliability 


^.e. , risk  is  discounted,  performance  is  estimated  optimistically,  <ipd  the 
warranty  option  price  becomes  k marketing  tool.  \ 

2A  great  deal  of  valuable  data'  acquired  during  reliability  demonstration  and 
similar  test  programs  had  simply  been  discarded  by  the  contractors  a:Tter  a 
few  years. 

3This  is  a key  point.  Hie  Services  have  developed  maintenance  reporting 
systems,  not  reliability  reporting  systems. 
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so  that  future  reliability  prediction  can  be  made  with  less 
uncertainty  and  greater  realism. 1 

E.  DIFFERENCES  BETWEEN  PROTOTYPE  AND  PRODUCTION  EQUIPMENT 

A number  of  the  potential  impacts  identified  in  Chapter  V 
are  related  to  expansion  of  present  development  programs  so 
that  more  testing  can  be  accomplished  on  equipment  that  is  as 
close  to  production  configuration  as  possible.  Many  of  the 
contractors  we  interviewed  believe  that  pilot  production  units  \ 

should  be  produced,  tested,  and  placed  into  service  prior  to 
pricing  a warranty.  In  their  experience,  prototype  equipment, 
assembled  and  tested  under  laboratory  conditions,  performs 
differently  (usually  better)  from  equipment  assembled  and 
acceptance  tested  in  quantity  production. 

The  significance  of  the  differences  observed  between  proto- 
type and  production  equipment  should  not  be  underestimated. 

Together  with  uncertainties  associated  with  the  field  operating 
environment,  the  uncertainties  associated  with  the  performance 
of  production  equipment  have  been  reported  to  be  responsible 
for  much  of  the  reliability  degradation  experienced  with  opera- 
tional equipment.2  And,  as  we  have  observed,  knowledge  of  field 
reliability  degradation  is  key  to  quantifying  contractor  risks 
and  to  pricing  warranty  options. 

These  factors  could  form  the  basis  for  a significant  devel- 
opment program  impact.  Pilot  production  or  low-rate  initial 
production  programs  might  be  included  as  a key  ingredient  in 


1 Whether  accurate  reliability  experience  extrapolations  can  be  made  in  view  of 
major  changes  in  technology  is  an  issue  yet  to  be  investigated.  We  have 
made  an  implicit  assumption  that  past  relationships  between  design  and 
demonstrated  and  realized  reliability  are  valid  indicators  of  future  ex- 
perience. This  hypothesis  needs  research,  especially  in  the  electronics 
technology  area. 

2After  differences  in  failure  definition  and  reporting  practices  have 
been  taken  into  account. 
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the  reliability  development  and  demonstration  process.  Pro- 
duction decisions  and  warranty  commitments  could  also  be 
delayed  until  the  results  of  the  initial  production  unit 
performance  are  known.  These  events  would  undoubtedly  have 
cost,  schedule,  and  contractual  impacts.  We  believe  that 
an  analysis  of  the  costs  and  benefits  of  implementing  these 
development  changes  on  selected  equipments  would  be  worthwhile. 

F.  DESIGN-TO-COST  VS.  DES I GN- FOR- WARRANTY 

As  noted  earlier,  a dichotomy  can  arise  whenever  contractors 
are  asked  to  design  to  meet  firm  unit-production  costs  goals, 
set  by  standards  of  acquisition  affordability,  and  then  are 
asked  to  design  equipment  with  guaranteed  operational  reliability. 
Our  research  found  that  important  trade-offs  affecting  equip- 
ment design,  production  cost,  and  field  reliability  can  be 
made  if  the  DTC  goals  are  mated  to  reliability  warranty  con- 
siderations in  the  initial  phases  of  the  engineering  development 
program.  Adding  a warranty  option  after  DTC  goals  have  been 
established  is  counterproductive  and  inhibits  the  potential 
beneficial  impacts  of  internalizing  reliability  and  maintain- 
ability responsibility  to  the  contractors.1 

G.  TECHNOLOGICAL  INNOVATION  VS.  DESIGN  CONSERVATISM 

There  is  not  much  doubt  that  the  possibility  of  warranty 
obligations  does  affect  the  contractor's  risk-taking  posture. 

One  area  of  impact  was  observed  in  the  use  of  advanced  tech- 
nology during  equipment  design.  Despite  many  potential  advan- 
tages to  the  utilization  of  advanced  components,  circuit 


lr[he  counterargument  to  these  perceived  difficulties  is  that  the  Government 
is  only  asking  the  contractors  to  stand  behind  the  specification  require- 
ments they  have  already  accepted;  therefore  DTC  and  warranties  should 
not  be  Incompatible  if  realistic  and  achievable  reliability-maintainability 
goals  have  been  accepted. 
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designs,  and  assembly  techniques,  the  use  of  unproven,  non- 
standard, or  low  product ion- volume  parts  was  reported  by  many 
to  be  inhibited.  The  ultimate  impact  was  believed  to  be  a 
tendency  toward  design  conservatism  on  warrantable  equipment  or 
more  thorough  design  proof-testing  during  the  advanced  develop- 
ment stage. 

H.  RETURN-ON-INVESTMENT  CONSIDERATIONS 

Warranties  have,  on  occasion,  been  described  as  a good 
business  venture  with  built-in  incentives  for  greater  contrac- 
tor profitability.  Possible  benefits  include  the  future 
profit  potential  as  a result  of  the  increased  reliability,  the 
availability  of  multi-year  warranty  funds  at  equipment  delivery 
rather  than  on  an  annual  basis,  and  the  opportunity  to  sustain 
equipment  manufacture  and  maintenance  during  the  warranty  period. 

To  achieve  these  monetary  and  other  returns,  most  contrac- 
tors we  interviewed  reported  than  an  additional  investment  beyond 
the  Government's  present  level  of  investment  in  engineering 
development  programs  would  be  required  to  reduce  the  risk  factor. 
Among  the  candidate  programs,  examples  of  this  type  of  invest- 
ment were  seen  in  the  form  of  company-funded  development  test 
and  demonstration  programs,  qualification  efforts  for  new  com- 
ponents and  parts,  revised  material  control  and  quality  assurance 
procedures,  additional  work  toward  adapting  prototype  designs 
for  production,  and  the  establishment  of  pilot  production  prior 
to  production  contract  award.  The  return  on  this  type  of 
investment  during  the  design  and  development  stage  was  reported 
to  be  acceptable  only  if  (a)  the  contractor  wins  the  production 
competition,  (b)  the  government  exercises  the  warranty  option, 
and  (c)  the  predicted  equipment  field  reliability  and  maintain- 
ability are  approached  in  practice. 

At  the  present  time,  many  contractors  we  interviewed  believe 
that  the  probability  of  all  these  events  occurring  is  such  that 
a reasonable  return  will  not  be  achieved. 
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I.  THE  COST  IMPACT  OF  RELIABILITY  WARRANTIES 

Our  research  was  concentrated  on  identifying  design  and 
development  program  impacts;  therefore,  we  did  not  explicitly 
examine  the  cost  impact  of  warranties  for  either  development 
or  equipment  production.1  Based  on  our  research,  however,  a 
cost  impact  can  be  expected  in  most  cases  for  additional 
development  effort.  This  cost  would  have  to  be  included  with 
the  equipment  warranty  price  to  estimate  the  full  cost  of  an 
equipment  warranty. 

Table  23  presents  a summary  of  the  estimated  warranty  cost 
impact  using  assigned  percentage  weightings  for  the  direct 
development  program  categories  and  the  impact  percentages  pre- 
sented in  Chapters  IV  and  V.2  This  table  illustrates  that,  if 
only  the  program  elements  of  the  contractors  reporting  impacts 
other  than  zero  are  analyzed,  the  direct  cost  impact  would  be 
11  percent  under  actual  or  beginning  warranty  application  condi- 
tions, 15  percent  if  the  warranty  requirements  were  firm  at  the 
beginning  of  engineering  development,  and  8 percent  if  specifi- 
cation relief  were  permitted.  If  all  program  element  impacts 
(including  zero)  are  used  for  analysis,  the  expected  cost  impact 
would  be  5 percent  for  the  actual  case,  8 percent  for  the  begin- 
ning case,  11  percent  for  the  firm  case,  and  5 percent  for  the 
specification  relief  case.  The  indirect  cost  elements,  materials 
and  purchased  parts,  financial  management,  value  engineering, 
and  contract  management,  have  not  been  included;  those  costs  were 
considered  to  be  assignable  to  overhead  or  indirect  accounts. 

The  actual  costs  of  achieving  predicted  field  reliability 
for  any  of  the  candidate  programs  are  not  known  at  this  time. 


^or  each  program,  the  relative  cost  contribution  by  functional  element  would 
be  necessary  to  calculate  the  R&D  cost  impact  from  data  reported  by  the  con- 
tractors. This  presents  an  interesting  topic  for  future  investigation. 

2The  assigned  percentage  weightings  have  been  selected  based  upon  the 
experiences  of  the  investigators.  Individual  development  programs  could 
differ  significantly  depending  upon  the  nature  of  the  development  task. 
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Case  I inpact  calculated  based  on  only  those  contractors  reporting  impacts  other  than  zero. 
Case  IT  impact  calculated  based  on  all  contractors  reporting  impacts,  including  zero. 


However,  we  believe  a valid  warranty  cost-benefit  study  would 
have  to  identify  and  estimate  the  non-recurring  direct  and 
indirect  costs,  as  well  as  the  recurring  costs  of  warranties, 
in  order  to  assist  in  the  warranty  decision  process. 

J.  THE  SUBSYSTEM  CONTRACTING  ENVIRONMENT 

The  unique  features  of  the  subsystem  acquisition  environ- 
ment have  been  docu’.iented  before,  especially  in  those  instances 
when  contract  requirements  were  dominant  forces  in  subsystem 
contractor  behavior. 1 Often,  subsystem  contractors  are  the 
subcontractors  of  major  prime  contractors,  as  is  the  case  for 
the  F-16  avionics.  And  while  the  subcontractors  may  be  obli- 
gated to  meet  or  exceed  the  major  provisions  of  their  prime 
contractor's  contract  with  the  Government,  they  have  little 
leverage  on  their  many  suppliers  to  pass  along  critical  contract 
risks  and  obligations. 

This  situation  of  risk  containment  prevailed  for  all  the 
candidate  subsystem  contractors  examined  in  this  study.  Not 
one  contractor  reported  that  he  was  able  to  pass  reliability 
warranty  guarantees  on  to  his  suppliers.  This  left  the  sub- 
system contractor  in  a position  of  weakness,  because  he  was 
unable  to  ensure  his  major  cost  components  or  spread  the  risk 
among  several  subcontractors.  This  factor,  among  others,  was 
important  in  establishing  risk-averting  management  behavior  and 
in  explaining  the  uncommon  concern  over  the  risk  implications 
of  the  warranty  obligations  that  we  observed  during  the 
interviews . 

K.  LEGAL  ENFORCEABILITY 

In  the  course  of  our  research,  we  encountered  frequent 


^or  a discussion  of  design-to-cost  acquisition  policies  and  the  subsystem 
contractor  environment,  see  C.  David  Weimer,  The  Application  of  Design- to- 
Cost  Acquisition  Policies  to  Selected  Electronics  Subsystem  Development 
Programs,  IDA  S-459  (1975),  PP-  67—73 • 
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discussions  about  the  potential  for  "gaming"  the  warranty 
option  such  that  potential  risks  are  not  encountered  in  actual 
practice  or,  if  they  are,  the  contract  can  be  modified  or 
voided.1  If  the  contractor,  under  a warranty  contract  situa- 
tion, believes  that  he  can  minimize  or  limit  his  potential  loss, 
then  his  basic  incentives  are  affected  from  early  in  the  design 
stage  through  production  and  deployment  of  the  equipment.  If 
the  risk-factored  profit  (expected  value  of  financial  gain)  is 
small  and  the  potential  penalty  or  loss  limited,  then  there  will 
be  little  real  incentive  to  expend  additional  time  and  effort 
early  in  the  development  program.  One  contractor  stated  that  it 
may  be  more  profitable  to  spend  resources  in  pursuit  of  legal 
remedies  than  in  product  improvement.  Moreover,  the  contractor's 
negotiating  leverage  increases  with  each  additional  unit  de- 
livered to  the  field,  particularly  because  he  will  be  dealing 
with  Service  management  who  did  not  participate  in  the  early 
contract  warranty  formulation. 

We  also  encountered  mention  of  the  ultimate  legal  enforce- 
ability of  warranty  penalties  often  enough  in  our  research  to 
believe  that  the  warranty  impact  on  the  design  and  development 
program  does,  in  fact,  depend  to  some  degree  upon  the  contrac- 
tor's assessment  of  both  the  Government's  policy  credibility, 
and  the  warranty's  eventual  legal  enforceability.2 

L.  CONCLUDING  REMARKS 

This  chapter  highlighted  a number  of  issues  that  either  in- 
dividually or  collectively  could  influence  the  equipment  design 


xIt  is  not  the  intent  to  discuss  the  strategies  and  methodologies  of  "gaming" 
in  this  report.  This  topic  is  presented  only  for  its  observed  and  potential 
impact  on  the  design  and  development  process  as  an  exogenous  impact  agent. 

2For  example,  a possible  legal  position  supported  by  a posture  of  uncon- 
scionability  resulting  frcm  the  market  power  of  the  Government  during 
negotiations  was  mentioned  during  our  research  interviews. 
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and  development  process.1  We  could  not  ascertain  whether  many 
or  all  of  these  issues  came  into  play  in  all  the  candidate 
programs.  However,  since  they  were  recurring  topics  of 
discussion,  initiated  primarily  by  contractor  personnel,  they 
have  been  raised  here  for  further  research  and  debate.  Some 
of  these  issues  could  be  determining  factors  affecting  the 
outcome  of  equipment  warranty  programs. 


3Preventing  an  impact  is  viewed  as  an  Important  Influence,  also. 
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SUMMARY  OF  FINDINGS  AND  CONCLUSIONS 
A.  SUMMARY  OF  FINDINGS 

The  research  effort  yielded  a number  of  significant  find- 
ings related  to  the  application  and  impact  of  warranties  on  the 
candidate  programs.  These  findings,  derived  from  the  question- 
naire responses  and  the  many  contractor  interviews,  are  sum- 
marized by  major  topic  below. 

1 . Warranty  Requirements  and  Their  Application 

Our  investigations  into  the  warranty  requirements  and  their 
application  provided  findings  in  areas  of  contract  requirements, 
contractor  development  environment,  equipment  maturity,  and 
warranty  negotiation. 

Most  of  the  programs  investigated  contained  multiple 
warranty  obligations  or  options  geared  to  the  achievement  of 
specified  operational  reliability.  The  requirement  details  were 
similar  for  all  programs  examined.  The  reliability  improvement 
warranty  and  guaranteed  MTBP  options  were  patterned  after  the 
Air  Force  Tacan  contract,  and  the  requirements  for  meeting 
Service  logistics  support  cost  targets  were  similar  to  those 
negotiated  for  the  Air  Force  UHF  radio  program. 

Equipment  warranties  covering  operational  performance  for 
time  periods  exceeding  two  years  were  found  to  be  new  contract 
requirements  for  the  candidate  contractors,  most  of  whom  lacked 
previous  military  warranty  experience.  The  warranties  therefore 
represented  a completely  new  contractual  commitment  for  the 
contractors . 
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Contractors  did  not  usually  negotiate  the  detailed  terms 
and  conditions  of  their  warranty  obligation  until  negotiation 
of  the  production  contract.  This  situation  was  found  to  inhibit 
important  design  trade-offs  during  the  engineering  development 
program. 

Most  (eleven  of  sixteen)  of  the  contractors  described  their 
equipment  as  a "new  technological  development"  rather  than  a 
product  improvement  over  a previous  model.  The  risk  associated 
with  predicting  the  ultimate  field  reliability  of  the  equipment 
was  found  to  be  increased  because  of  the  lack  of  relevant  field 
data  on  previous  equipment. 

Difficulties  or  critical  areas  during  warranty  require- 
ments negotiation  were  reported  in  eleven  major  categories. 
Measurement  of  field  reliability,  definition  of  relevant  fail- 
ures, and  warranty  price  were  the  critical  negotiating  areas 
most  frequently  identified  by  the  contractors. 


2.  Impact  on  Candidate  Development  Programs 

The  overall  impact  of  warranty  requirements  on  the  candi- 
date development  programs,  in  terms  of  the  increase  in  develop- 
ment program  effort,  was  reported  to  be  small.  An  average 
impact  of  5*5  percent  increase  in  effort  was  recorded  based  on 
all  sixteen  questionnaire  responses.  The  development  program 
functional  elements  receiving  the  greatest  impact  were  contract 
management,  program  management,  design  analysis,  data  and 
reports,  and  financial  management. 

Several  requirements  were  identified  as  influencing  the 
execution  of  the  development  program.  Explicit  requirements  for 
equipment  seals,  elapsed- time  indicators,  and  environmental 
and  operational  data  directly  influenced  the  functions  of 
design,  prototype  manufacture,  and  report  preparation, 
respectively.  Implicit  equipment  performance  and  maintaina- 
bility requirements  for  MTBP,  MTTR,  and  TAT  influenced  efforts 
in  program,  contract,  and  financial  management  as  a part  of 
the  overall  risk  assessment. 
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Major  barriers  to  additional  program  impacts  were  also 
identified.  The  major  constraints,  as  reported  by  the  contrac- 
tors, were  the  uncertainty  of  warranty  requirements,  the 
shortage  of  additional  development  funds,  the  constrained  de- 
velopment program  schedule,  preset  unit-production  cost  goals, 
and  inflexible  equipment  performance  and  product  specifications. 


3.  Impact  on  Product  Designs 

The  impact  of  warranty  options  on  equipment  design  was 
reported  to  be  minor,  except  for  contract-imposed  requirements 
for  box  seals,  elapsed-time  indicators,  and  warranty  labels. 

A small  impact  was  reported  in  the  areas  of  built-in  test 
circuitry  or  other  means  of  failure  isolation,  such  as  test 
equipment  accommodation. 

The  primary  barrier  to  design  impacts  appeared  to  be  the 
requirement  to  design  equipment  suitable  for  both  Service 
organic  maintenance  and  contractor  maintenance . 


4.  Potential  Impact  on  Candidate  Development  Programs 

Three  warranty  application  conditions  that  could  lead  to 
increased  warranty  impact  were  identified  and  investigated 
during  the  research.  These  conditions  were:  (1)  negotiation 

of  warranty  terms  and  conditions  at  the  beginning  of  engineer- 
ing development,  (2)  elimination  of  the  option  status  of  war- 
ranty contract  requirements,  and  (3)  greater  specification 
flexibility. 

The  responses  to  the  IDA  research  questionnaire  indicated 
that  the  overall  development  program  effort  would  have  been 
increased  from  5.5  percent  to  8.2  percent  if  the  negotiation 
of  warranty  terms  and  conditions  had  been  accomplished  prior  to 
the  beginning  of  engineering  development;  similarly,  the  program 
effort  would  have  been  increased  to  10.5  percent  if  the  option- 
decision  uncertainty  had  been  removed.  If  greater  specification 
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flexibility  were  also  permitted,  total  additional  effort  would 
increase  nominally,  to  5.6  percent.  However,  the  areas  for 
increased  effort  would  be  different  from  those  in  the  current 
programs . 

5.  Correlation  of  Contractor  Responses 

Certain  contractors  consistently  reported  greater  increases 
in  development  program  effort  than  did  other  contractors  over 
all  conditions  examined.  For  those  contractors  reporting  a 
warranty  impact  other  than  zero,  the  estimated  impact  on  the 
candidate  development  program  averaged  12.4  percent.  The 
reported  impacts  for  the  beginning , firm,  and  specification 
relief  conditions  examined  in  Chapter  V were  12.8,  15.1,  and 
9.9  percent,  respectively.  Except  for  the  actual  warranty 
application  case,  this  contractor  subset  reported  an  average 
impact  that  was  60  percent  higher  than  the  total  aggregate 
sample . 

Attempts  to  correlate  this  group  of  contractors  with  other 
contractor  characteristics  were  unsuccessful.  Attempts  to 
identify  relationships  between  contractor  response  and  program 
status.  Service  sponsor,  competitive  environment,  major  system 
application,  and  other  characteristics  were  also  unsuccessful. 

No  identifiable  rationale  for  specific  contractor  behavior 
could  be  found. 

6.  Additional  Issues  Relating  to  Warranty  Impacts 

The  following  additional  issues  relating  to  warranty  im- 
pacts were  identified  during  the  research  effort: 

• Contractor  Motivation  for  Increased  Reliability 

• Government  Credibility  for  Warranty  Success 

• Competitive  Environment 

• Prediction  of  Field  Reliability 

• Differences  Between  Prototype  and  Production  Equipment 
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• Design-to-Cost  vs.  Design-for-Warranty 

• Technological  Innovation  vs.  Design  Conservatism 

• Return-on-Investment  Considerations 

• Cost  Impact  of  Equipment  Warranties 

• Subsystem  Contracting  Environment 

• Legal  Enforceability  of  Warranty  Contracts 

These  issues,  either  individually  or  in  the  aggregate,  were 
reported  to  influence  contractor  behavior  during  engineering 
development  by  helping  to  create  a greater  impact  or  by  pre- 
senting barriers  that  precluded  further  impact. 

B.  CONCLUSIONS 

Based  on  the  findings  of  the  limited  research  effort,  the 
following  conclusions  can  be  drawn  regarding  warranty  impacts 
on  the  candidate  subsystems  examined  during  the  study. 

1 . Warranty  Application  Has  Been  Uncertain  and  Imperfect 

The  application  of  warranties  to  the  candidate  electronics 
subsystem  development  programs  was  made  in  an  environment  of 
uncertainty  created  by  the  experimental  nature  of  the  warranty 
acquisition  policy,  the  lack  of  previous  contractor  experience 
with  warranties,  the  advanced  technological  status  of  the  equip 
ment,  and  the  lack  of  definitized  warranty  requirements  during 
development.  This  environment  and  contractor  uncertainty  about 
the  Government's  ultimate  exercise  of  the  warranty  option  re- 
sulted in  an  imperfect  application  of  warranty  techniques. 
Because  of  these  conditions,  we  believe  the  warranty  experiment 
represented  by  the  candidate  subsystems  may  not  be  conclusive 
indicators  of  the  success  or  failure  of  the  Government's 
warranty  policy. 
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2. 


Warranty  Options  Are  Not  Yielding  Design  and  Development 
Incentives  for  Improved  Reliability 

Given  the  uncertainties  surrounding  the  application  of  war- 
ranties and  the  observed  impact  on  the  candidate  programs,  it 
is  concluded  that  warranty  options  are  not  changing  the  equip- 
ment design  and  development  process  in  ways  that  would  either 
significantly  improve  the  inherent  reliability  of  the  equipment 
or  provide  necessary  additional  information  to  contractors 
regarding  expectations  of  future  equipment  field  reliability. 
Presently,  the  candidate  warranty  options  can  be  better 
characterized  as  fixed-price  options  for  future  maintenance  of 
production  equipment. 

3.  The  Potential  Exists  for  Increased  Warranty  Impact 

There  is  a potential  for  increased  beneficial  warranty  im- 
pacts on  the  equipment  design  and  the  development  process  if 
uncertainties  and  barriers  surrounding  warranty  application  are 
removed.  Elimination  of  the  optional  nature  of  present  warranty 
programs,  for  example,  appears  to  be  an  important  step.  The 
increased  impact  can  be  achieved  most  efficiently  if  maximum 
allowable  specification  relief  accompanies  changes  in  warranty 
policy.  Realization  of  the  increased  impacts  could  necessitate 
longer  develooment  schedules  and  additional  development  funds 
because  of  the  increased  effort  involved.  However,  we  believe 
that  this  investment  would  be  cost-effective  over  a subsystem's 
life  cycle  if  equipment  reliability  and  availability  were  there- 
by significantly  improved. 

4.  Warranty  Options  Have  Captured  Management  Attention 

The  interviews  and  questionnaire  responses  have  demonstrated 
that  warranty  options  have  attracted  the  attention  and  concern 
of  Service  and  contractor  management.  This  concern  has  been 
prompted  by  the  great  uncertainties  surrounding  the  candidate 
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program  outcomes.  Steps  to  translate  this  concern  Into  positive 
action  for  improved  equipment  reliability  are  potentially 
3-V3-il.&ble  and  are  outlined  as  "Guidelines  for  Warranty  Policy 
Development  and  Application”  in  Chapter  IX. 


IX 


GUIDELINES  FOR  WARRANTY  POLICY  DEVELOPMENT  AND  APPLICATION 

The  findings  derived  from  Government  and  contractor  inter- 
views and  the  data  retrieved  during  this  research  study  support 
several  guidelines  for  continued  warranty  policy  development. 
These  guidelines,  presented  below,  are  steps  that  OSD  and  the 
Services  should  consider  to  improve  the  application  of  warranties 
to  electronic  subsystem  development  programs.  In  particular, 
these  guidelines  are  aimed  at  enriching  the  design  and  develop- 
ment process  and  providing  greater  leverage  over  life-cycle 
costs  and  better  operational  performance  for  the  development 
funds  expended. 

A.  OPTIMAL  SCHEDULING  FOR  WARRANTY  DECISIONS 

In  order  to  achieve  maximum  benefits  from  equipment  war- 
ranties, the  maintenance  philof^ophy  for  the  candidate  equipment 
should  be  established  by  the  completion  of  the  equipment’s  ad- 
vanced development  stage,  i.e.,  before  DSARC  II.  The  option 
process  for  exercising  warranties  should  be  discarded  as 
counterproductive  to  optimizing  equipment  design  and  to  achiev- 
ing increased  equipment  reliability  and  maintainability  through 
the  development  process. 

B.  WARRANTY  REQUIREMENTS  DEFINITION 

The  specifications  and  requirements  for  future  equipment 
warranties  should  be  negotiated  early  in  the  equipment  develop- 
ment program,  preferably  during  negotiation  of  the  engineering 
development  contract.  These  requirements  should  be  treated 
the  same  as  equipment  design  or  performance  specifications. 
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c. 


INTEGRATION  OF  WARRANTIES  WITH  DESIGN-TO-COST  PROGRAMS 


The  application  of  warranties  to  programs  that  already  con 
tain  design-to-cost  goals  should  initiate  a reassessment  of  the 
production  cost  targets  to  minimize  the  sum  of  future  acquisi- 
tion and  operational  costs.  A formal  process  should  be  estab- 
lished to  enable  contractors  and  Service  project  officers  to 
revise  unit-production  cost  goals  if  such  a change  can  be  justi 
fled  on  the  basis  of  reduced  life-cycle  costs. 

D.  SPECIFICATION  FLEXIBILITY 

Specification  flexibility  for  non-critical  requirements 
assumes  a role  of  even  greater  importance  when  contractors  are 
required  to  assume  additional  responsibilities  for  equipment 
performance.  Thus  specification  flexibility  should  be  an 
integral  part  of  each  warranty  procurement. 

E.  RELIABILITY  DATA  AND  PREDICTIVE  MODELS 

To  aid  in  improving  reliability  prediction  retrieval  of 
current  and  past  field  reliability  data  on  all  generic  equip- 
ment suitable  for  warranty  application  should  be  continued. 
Efforts  also  should  be  increased  to  predict  more  accurately 
future  field  reliability  from  demonstration  test  data.  These 
initiatives  should  be  taken  with  help  from,  and  feedback  to, 
the  industrial  contractor  sector. 

F.  DEVELOPMENT  FUNDING  AND  SCHEDULE  FLEXIBILITY 

Preparations  for  meeting  equipment  warranty  guarantees  may 
require  additional  development  time  and  funds.  The  Defense  De- 
partment should  recognize  this  possibility  and  take  steps  to 
ensure  that  additional  time  and  money  are  available. 
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TOPICS  FOR  ADDITIONAL  RESEARCH 


Most  research  studies  uncover  a number  of  questions  that 
are  not  answered  by  the  original  effort,  and  this  study  is  not 
an  exception.  Many  questions  that  were  surfaced  during  our  in- 
vestigation demand  research  attention  if  the  application  experi- 
ence with  warranties  is  to  mature.  The  more  important  subject 
areas  for  additional  research  are  listed  below;  all  are  presented 
in  question  form  for  the  benefit  of  prospective  investigators. 

■A 

• How  cost  effective  are  equipment  warranties  when 
all  direct  and  indirect  costs  and  benefits  are 
considered? 

• Have  warranties  affected  the  engineering  change 
proposal  (ECP)  process  and  application  experience? 

• What  impact  will  widespread  use  of  warranties 
exert  on  the  Services’  logistics  organizations 
and  operations? 

• What  is  the  cost  value  of  uncertainties  in  the 
prediction  of  future  field  reliability  and  main- 
tainability? 

• What  are  the  historical  trends  and  supporting 
methodologies  for  predicting  reliability? 

• Do  warranty  considerations  inhibit  technological 
innovation  during  design  and  development? 

• What  has  been  the  actual  field  operating  experience 
to  date  with  the  multi-year  warranties? 

• Are  warranties  for  parts  of  subsystems  feasible? 

What  are  possible  ramifications  of  a partially 
warranted  weapon  system? 
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DEFENSE  ADVANCED  RESEARCH  PROJECTS  AGENCY 

1-too  wilson  moulevaro 

ARLINGTON.  VIRGINIA  22209 


TASK  ORDER  FOR  WORK  TO  BE  PERFORMED 

BY 

INSTITUTE  FOR  DEFENSE  ANALYSES 


TASK  ORDER  T- 17  5 


DATE : 


You  arc  hereby  requested  to  undertake  the  following  task: 

1.  TITLE : Reliability  Improvement  Warranty  Impact  Assessment 

2 . I W\  CKO  ROUND : Reliability  Improvement  Warranties  (RIW)  and 

other  reliability  guarantees  offered  by  equipment  manufacturers 
have  proven  to  be  effective  ways  for  improving  equipment 
operational  reliability  in  commercial  airline  applications. 

Based  upon  the  commercial  experience  with  avionics  equipment 
during  the  past  ten  years,  the  purchase  of  RIWs  with  equipment 
is  now  a standard  and  cost-effective  practice  for  airline 
avionics  procurement. 

Extended-duration  warranties  for  military  application  were 
initiated  by  the  Navy  in  1969  with  the  acquisition  of  a Lear- 
Seiglcr  airborne  gyro  assembly.  Subsequent  development  of  the 
concept  has  led  to  widespread  consideration  of  RIW  options  for 
avionics  equipment  in  all  Services. 

RIWs  have  also  become  important  ingredients  in  the  Design- to- 
Gost  (DIG)  acquisition  process  where  total  life-cycle  costs  are 
critical  considerations.  In  a recent  study,  IDA  has  documented  , 
l lie  application  of  RIW  options  to  several  electronics  subsystems. 
This  investigation  indicated  that  the  incorporation  of  RIW  options 
or  reliability  r.uaran tecs  into  equipment  acquisition  programs 
could  significantly  influence  the  design  and  development  process 
and  the  developed  equipment.  The  impact  of  RIW  options  upon 
equipment  acquisition  was  identified  as  a topic  needing  further 
s tudy . 


IDA  Study  G-/i59,  "The  Application  of  Dosign-to-Cost  Policies 
to  Selected  Electronics  Subsystem  Development  Programs", 

June  1975. 


A-l 


erf 


TASK  ORDER  T-125 


3.  OBJECTIVE : The  objective  of  this  task  is  to  analyze  the 

impact  of  RIW  options  upon  the  design  and  development  process 
and  the  resulting  systems  and  subsystems. 

4.  SCOPE : IDA  is  asked  to  investigate  and  analyze  Service 

experiences  to  date  in  the  planning  and  execution  of  develop- 
ment programs  containing  provisions  for  RIW  options.  The 
programs  selected  for  study  with  DDR&E  concurrence  will 
consider  as  candidates  those  electronic  subsystems  previously 
investigated  for  Design- to-Cost  policy  application  and 
selected  avionics  subsystems  being  developed  for  the  multi- 
national F-16  Air  Combat  Fighter  (ACF)  as  well  as  others 
appropriate  to  this  task.  The  depth  of  the  analysis  will 
depend  upon  the  extent  to  which  appropriate  data  and  information 
on  the  systems  can  be  obtained.  For  each  program  (within  the 
scope  of  the  available  data) , the  requirements  of  the  RIW  option 
will  be  documented,  and  the  impact  of  these  requirements  upon 
development  program  planning,  equipment  design,  development 
testing,  development  program  cost  and  schedule,  and  equipment 
production  cost  and  performance,  will  be  analyzed. 

5.  SPECIFIC  TASKS: 

1.  Identify  and  document  for  each  candidate  program  the 
requirements  for  RIWs  which  were  established  and  imposed 
upon  the  program  during  the  development  phase. 

2.  Document  and  analyze  the  impacts  of  the  reliability 
specification  and  RIW  requirements  upon  the  design  and 
development  process  in  terms  of  development  program 
scope,  cost  and  schedule,  and  product  design,  perfor- 
mance and  cost,  form,  fit  and  function,  and  use  of 
Government  Furnished  Equipment  (GFE) . 

3.  Based  upon  data  and  insights  developed,  Identify  steps 
that  OSD  and  the  Services  can  take  in  the  development 
process  to  assure  that  newly-developed  items  can 
transition  to  production  with  warranty  provisions  which 
assign  commensurate  risks  to  both  the  Government  and 
contractors . 

6.  SCHEDULE : The  first  phase  of  this  task  will  be  limited  to 

45  days  for  the  purpose  of  identifying  and  selecting  the 
candidate  programs.  Thereafter  informal  progress  report 
briefings  are  requested  at  three-month  intervals  during  the 

task  effort.  A Draft  Final  Report  is  requested  on  1 October  1976. 
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7.  ODDR&E  COGNIZANCE:  Cognizance  of  this  task  is  within 

the  Office  of  the  Assistant  Director  (Planning),  ODDR&E . 

I 

8.  SCALE  OF  EFFORT:  A maximum  of  15  man-months  of  effort, 

including  use  of  consultants  is  authorized  for  this  task. 

9.  REPORT  DISTRIBUTION  AND  CONTROL : All  report  distribution 

will  be  controlled  by  the  office  of  technical  cognizance. 

10.  SPECIFIC  INSTRUCTIONS  AND  LIMITATIONS:  None.  Changes 

in  scales  of  effort  will  not  be  made  without  the  consent  of 
ARPA.  A "need-to-know"  is  hereby  established  in  connection 
with  this  task  and  access  to  U.S.  and  foreign  program 
information  in  the  field  of  this  Task  is  authorized  for 
participating  personnel  and  such  supervisory  and  advisory 
personnel  as  deemed  necessary.  Department  of  Defense  support, 
such  as  access  to  classified  documents  and  publications, 
security  clearances,  and  the  like,  necessary  to  complete 

this  Task,  will  be  obtained  through  the  cognizant  ODDR&E 
office . 


George  II.  Heilmeier 
Director 


ACCEPTED: 


"Alexander  H'.  'Flax 
President,  IDA 


Lf 


DATE:  10  October  1975 
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Appendix  B 

AFSC  MEMORANDUM:  SUPPORT  FOR  TRI-SERVICE  RELIABILITY 
AND  SUPPORT  INCENTIVES  WORKING  GROUP 


• a.  \ k . 


DEPART”  ”!  f;'1  Z * M*  T-'^r: 

HEADQUARTER  j .C.S  ?q:  v,  r-vi^.iS  COMMAND 

ANDREWS  AT2,  ffaSCIlSCtfCN,  D.C.  20334 

&)  " ore  1975 

Support  for  Trl- Service  Reliability  and  Support  Incentives 
Working  Group 


ASD/YP/TPE/YPL/rwv  /YllK 


ts?/  ocio 


1.  Af SC  is  participating  in  a Tri -Service  Reliability  and  Support 
Incentives  Working  Group  recently  chartered  by  the  Acting  Assistant 
Secretary  of  Defense  (I6L)  and  DbRQE.  The  prime  purposes  of  this 
Group  are  to  develop  improved  policies  and  procedures  for  motivating 
contractors  to  produce  reliable  systems  and  to  enhance  the  transfer 
of  lessons  learned  in  Reliability  Improvement  Warranties  and  Life 
Cycle  Costing. 

2.  The  Institute  for  Defense  Analyses  (IDA)  has  been  tasked  to 
support  the  Tri -Service  Working  Group  in  achieving  its  objectives. 
IDA  will  study  our  experienco  with  the  various  incentive  techniques 
and  their  impact  upon  the  development  of  electronics  subsystems. 

The  IDA  study  director  is  Dr.  David  Woioer,  AUTOVCN  225-1382. 

3.  Your  office  is  requested  to  cooperate  with  tho  IDA  study  team 

in  providing  access  to  the  appropriate  co  itractors  and  program  data. 

FOR  THE  COMMANDER 

S "rrji*!Q 


MELVIN 

'.S&i-JtlV*. 


Colonel,  USAE 


Cy  to:  AFLC/AQf'lLb 
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OFFICE  OF  THE  SECRETARY  OF  DEFENSE 

WASHINOTON,  D.C.  10  JO  I 

4 NOV  875 


MEMORANDUM  FOR  Director  of  Defense  Research  and  Engineering 

Assistant  Secretary  of  Defense  (IScL) 

Assistant  Secretaries  of  the  Military  Departments  (R&D) 
Assistant  Secretaries  of  the  Military  Departments  (I&L) 

SUBJECT:  Tri-Service  Reliability  and  Support  Incentives  Working  Group 

The  tri-Service  group  which  has  been  formed  as  a result  of  our  memorandum 
of  5 September  1975  (same  subject)  has  received  excellent  support  from  the 
Services.  The  first  meeting  of  the  working  group  was  held  7 October  197  5 
and  as  you  can  see  from  the  attached  minutes  and  action  plan  there  are  a 
significant  number  of  critical  tasks  to  be  accomplished.  Continued  emphasis 
will  be  placed  on  identifying  and  developing  guidelines  that  will  have  an  early 
pay-off.  The  working  group  activities  will  be  focused  on  tasks  related  to: 

(1)  Risk/Cost  Analysis,  (2)  Application  of  RIW's,  and  (3)  Other  Reliability 
and  Support  Cost  Incentives. 

As  a result  of  your  favorable  support  of  this  endeavor,  assurance  has  been 
received  from  both  Army  and  Navy  that  $55,  000  each  will  be  made  available 
to  Air  Force  for  the  Tri-Service  RIW  Data  Base  and  Analysis  Tasks.  It  is 
anticipated  that  the  Services  will  identify  their  task  requirements  by 
6 November  1975,  the  Air  Force  will  translate  these  requirements  to  a firm 
statement  of  work  by  19  December  1975  and  a contract  could  be  awarded 
during  February  1976. 


The  next  meeting  of  the  working  group  will  be  held  6 November  1975.  At 
that  time  the  Service  working  group  members  are  to  present  how  the 
independent  analysis  of  RIW  applications  is  conducted  in  response  to  the 
ASD(I&L)  memorandum  of  16  September  1975,  (Subj:  Reliability  Improve- 
ment Warranty  (RIW)  Guidelines). 

It  appears  that  production  phase  RIW  requirements  will  influence  the 
research  and  development  (R&D)  process  and  result  in  improved  equipment 
reliability.  The  full  R&D  impact  of  these  requirements  has  yet  to  be 
assessed  and  is  therefore  a task  area  of  the  working  group.  Accordingly, 
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the  Institute  for  Defense  Analysis  (IDA)  has  been  tasked  to  observe,  record, 
and  analyze  the  impact  RIWs  have  on  the  development  process  for  those 
programs  previously  examined  for  Design  to  Cost  (DTC)  policy  application. 
Also,  they  are  to  identify  other  equipment  developments  where  RIW  can  be 
implemented  to  affect  the  development  process  and  where  impact  can  be 
assessed.  For  the  IDA  project  leader,  Dr.  C.  David  Weimer,  to  perform 
effectively  during  the  study,  access  to  information  from  Service  staff, 
program  manager,  and  contractor /subcontractor  offices  is  essential.  With 
your  concurrence  the  Service  staff  members  of  the  working  group  will  be 
considered  as  the  points  of  contact  for  the  purposes  of  this  study. 

Your  support  of  the  working  group  and  the  application  of  RIW's  to  reduce 
support  cost  and  improve  field  reliability  of  equipment  is  appreciated. 


ROBERT  E.  BERRY 


Deputy  Director 
(Policy  & Planning) 


-Q^jacques  s.  gansler 

Deputy  Assistant  Secretary  of  Defense 
(Materiel  Acquisition) 
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400  ARMY-NAVY  DRIVE,  ARLINGTON,  VIRGINIA  22202  • TELEPHONE  (703)  558-1000 


IDA 

INSTITUTE  FOR 
DEFENSE  ANALYSES 

To:  Distribution 

Subject:  Reliability  Improvement  Warranty  Questionnaire 


The  Institute  for  Defense  Analyses  (IDA)  has  been  directed  by  DDR&E  (Office 
of  the  Deputy  Director  for  Policy  and  Planning)  to  Investigate  the  Introduc- 
tion of  Reliability  Improvement  Warranties  (RIWs)  and  other  life-cycle  cost 
guarantees  Into  recent  defense  electronics  acquisitions.  The  focus  of  the 
IDA  effort  is  identification  of  the  Impact  that  these  new  initiatives  may 
have  upon  future  equipment  development  programs  and  equipment  design. 

Recent  IDA  research  in  defense  electronics  acquisition  (IDA  Study  Report 
S-459)  has  revealed  that  requirements  for  equipment  warranties  or  life- 
cycle  cost  guarantees  can  be  coupled  to  the  design  of  the  equipment  and  the 
conduct  of  the  engineering  development  program.  Many  questions  or  problem 
areas  associated  with  implementing  operational  guarantees  are  best  addressed 
by  taking  appropriate  actions  during  the  design  and  development  process 
thereby  reducing  risk  and  uncertainty  for  both  defense  contractors  and  the 
Government. 

The  purpose  of  the  IDA  research  task  is  to  identify  both  beneficial  as  well 
as  dysfunctional  Impacts  of  equipment  operational  performance  guarantees 
upon  the  electronics  development  process.  The  task  is  also  aimed  to  delineate 
potential  design  and  development  barriers  which  impede  the  effective  applica- 
tion of  these  guarantees. 

To  achieve  these  objectives,  nineteen  development  contractors  representing 
fourteen  candidate  electronics  programs,  as  shown  in  Attachment  A,  are 
being  Interviewed  and  are  requested  to  complete  the  attached  questionnaire 
for  their  particular  program.  Additional  interviews  will  subsequently  be 
scheduled  to  retrieve  the  questionnaire  and  discuss  specific  candidate 
program  Issues.  The  responses  of  all  contractors  will  be  summarized;  all  data 
will  be  coded  to  prevent  disclosure  of  specific  programs  or  contractors. 

The  output  of  the  research  effort  will  be  an  identification  of  changes  which 
can  be  made  during  the  design  or  development  process  to  favorably  affect  the 
successful  negotiation  and  Implementation  of  product  field-performance 
guarantees.  Potential  barriers  to  achievement  of  lower  life-cycle  cost  and 
Increased  field  reliability  will  also  be  Identified  for  Government  require- 
ments review. 

Copies  of  the  resulting  study  report  will  be  sent  to  each  respondent  as  well 
as  all  candidate  program  offices. 
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II  have  questions  or  need  clarification  regarding  the  questionnaire  or 

^7no!D?cltVoLeff0rt’  you  are  encouraged  to  contact  the  undersigned  at 
\/U3/  558-1843. 

Thank  you  for  your  cooperation  and  participation  in  this  project. 


Attachments 

(A)  List  of  candidate  subsystems  and  contractors. 

(B)  RIW  Impact  Questionnaire 
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CANDIDATE  SUBSYSTEMS  FOR  RIW  IMPACT  ASSESSMENT  STUDY 
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THE  IMPACT  OF  RELIABILITY  IMPROVEMENT  WARRANTIES 

UPON 

ELECTRONICS  EQUIPMENT  DESIGN  AND  DEVELOPMENT 


CANDIDATE  PROGRAM: 


PROJECT  PROPRIETARY  INFORMATION 
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THE  IMPACT  OF  RELIABILITY  IMPROVEMENT  WARRANTIES 

UPON 

ELECTRONICS  EQUIPMENT  DESIGN  AND  DEVELOPMENT 
INTRODUCTION 

This  questionnaire  is  designed  to  solicit  the  experiences 
and  the  opinions  of  selected  defense  contractors  who  are 
currently  or  have  recently  engaged  in  development  programs 
containing  requirements  for  future  equipment  Reliability 
Improvement  Warranties  (RIWs).  The  primary  purpose  of  the 
questionnaire  is  to  identify  the  actual  or  potential  impact 
that  RIWs  or  RIW  options  have  upon  the  development  process  and 
the  developed  equipment.  A subsidiary,  but  critically  important, 
objective  of  this  questionnaire  is  to  identify  those  elements  of 
current  electronics  development  programs  which  inhibit  or  act  as 
barriers  to  the  success  of  RIW  implementation. 

While  it  is  the  intent  of  this  questionnaire  to  derive 
information  acquired  on  a specific  program  managed  by  your 
company,  the  results  of  the  questionnaire  will  be  combined  in 
such  a manner  as  to  conceal  all  program  identification.  Your 
response  will  be  held  in  strictest  confidence;  no  correlation 
or  identification  will  be  made  in  the  reported  results  between 
responses  and  specific  programs.  Data  from  each  of  the  nineteen 
selected  contractors  will  be  held  by  IDA  as  Company-Proprietary 
information. 

This  questionnaire  is  divided  into  four  parts.  Part  A is 
designed  to  provide  general  background  information  concerning 
your  company  and  your  past  experience  with  product  warranties. 
Part  B solicits  information  concerning  your  recent  or  current 
development  program  and  the  equipment  under  development.  Part 
B also  solicits  your  opinions  and  suggestions  for  Improvements 
in  the  development  process.  Part  C addresses  the  construction 
of  a proposed  engineering  development  program  under  assumed 


conditions  that  an  RIW  is  not  optional  but  is  a pre-planned 
condition  for  future  equipment  procurement.  Part  D is  an 
attempt  to  obtain  specific  examples  of  ^he  RIW  impact  upon 
development  programs  and  equipment . 

Most  of  the  questions  contained  in  this  questionnaire  can 
be  answered  by  simply  checking  the  appropriate  box  or  by  supply- 
ing a few  words  relative  to  your  particular  program.  However, 
it  is  recognized  that  all  of  the  key  Issues  surrounding  RIWs 
and  the  development  process  may  not  be  identified  by  the 
questions.  Therefore,  sufficient  space  has  been  provided  for 
your  use  to  amend  or  elaborate  upon  the  questions  asked.  This 
type  of  "feedback"  is  very  Important  to  this  study  and  is 
specifically  encouraged. 
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BACKGROUND  INFORMATION: 


Title  or  position  of  respondent.  

Briefly  indicate  past  development  or  production  programs 
which  contained  warranties  or  warranty  options.  


The  candidate  program  can  be  characterized  as: 

/~7  A Product  Improvement  (Over  A Previous  Model) 

/_/  A New  Technological  Development 

£7  

As  of  January  1,  1976,  the  status  of  the  candidate  product 
development  program  was: 

/~7  10%  Complete  /~7  25%  Complete  /~7  50$  Complete 
/~7  Completed 

The  topic  of  an  RIW  warranty  was  first  formally  introduced 
to  the  candidate  development  program: 

/~7  With  the  Development  RFP  /~7  Prior  to  PDR 

/~7  Prior  to  CDR  /~7  Prior  to  Qual.  Testing  or 

Prototype  Demonstration 

/~7  With  the  Production  RFP 

At  what  point  were  the  terms  and  conditions  of  the  RIW 
option  negotiated? 

\ 

/~7  Development  Contract  Award  /~7  CDR 

/~7  Production  RFP  /~7  Production  Contract  Award 

/~7  Not  Yet  Negotiated 
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Indicate  the  most  critical  areas  for  negotiation  you 
believe  are  contained  in  your  RIW  option. 

/~7  Definition  of  Relevant  Failures 

/~7  Turn-Around  Time 

/~7  Consignment  Spares  Provisions 

/~7  Warranty  Price 

/~7  Measurement  of  Field  Reliability 
/~7  ECP  Procedures 
/~7  Economic  Escalation  Provisions 
/~7  Configuration  Control  Provisions 
/~~7  Contractor  Data  and  Reports 
/~7  Warranty  Time  Period 

/~7  Other  (Specify)  


What  is  (or  was)  the  approximate  dollar  value  of  the 
candidate  equipment  development  program?  $ 


B. 


IMPACT  OF  RIM  OPTION  ON  THE  CURRENT  OR  RECENT 
DEVELOPMENT  PROGRAM 


1.  Listed  in  the  table  below  are  the  functional  elements 
of  a typical  electronics  development  program.  In  the  first 
column  you  are  requested  to  identify  those  elements  which  were 
actually  influenced  by  the  presence  or  possibility  of  the  RIW 
option  and  estimate  the  impact  in  terms  of  incremental  effort 
or  program  scope  change.  In  the  second  major  column  you  are 
requested  to  indicate  your  best  estimate  of  what  the  impact  of 
the  RIW  option  would  have  been  if  the  terms  and  conditions  had 
been  negotiated  prior  to  start  of  the  engineering  development 
program.  Assume  for  both  cases  that  the  warranty  is  to  be  a 
priced  option  and  that  Government  organic  maintenance  require- 
ments were  also  imposed  upon  the  program.  Do  not  hesitate  to 
add  to  the  functional  list  or  make  other  appropriate 
modifications. 

The  choice  of  response  for  each  functional  element  has  been 
divided  into  six  possibilities  ranging  from  a moderate  decrease 
in  effort  to  a significant  Increase  in  total  effort.  Assume 
that  a "small”  increase  or  decrease  represents  approximately 
a 10  percent  change,  a "moderate"  increase  or  decrease  a 20 
percent  change,  and  a "significant"  increase  represents  any- 
thing over  25  percent  additional  effort. 
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TABLE  1 


DEVELOPMENT  PROGRAM  ELEMENT 
DESIGN 

CIRCUIT 

CARD  AND  BOARD 

NODULE 

SRA 

LRU  OR  FLU 

SYSTEM  ARCHITECTURE 

BUILT-IN  TEST 

TEST  EQUIPMENT 

GROUND  SUPPORT  EQUIPMENT 

PRODUCTION  TOOLING 

ANALYSIS 

THERMAL 

STRESS 

EMC 

ENVIROIWENT 

FAILURE-MODE-EFFECTS 

RELIABILITY 

MAINTAINABILITY 

PROOUCIBILITY 

OPTIMUM  REPAIR  LEVEL 

COST  - PERFORMANCE 

MATERIALS  AND  PURCHASED  PARTS 

PART  SELECTION 

VENDOR  SCREENING 

COST  ESTIMATING  ANO  PRICING 

PROTOTYPE  MANUFACTURE 

COMPONENT  PROCUREMENT 
CARO  OR  BOARO  ASSEMBLY 
MODULE  ASSEMBLY 
BOY  OR  CHASSIS  ASSEMBLY 
FUNCTIONAL  TEST  AND  CHECKOUT 

EQUIPMENT  TEST 
OART/COMPONENT  ACCEPT.  TEST 
BREADBOARD  TESTS 
BRASSBOARO  TESTS 
EQUIP.  PERFORMANCE  OEMON. 
EQUIPMENT  ENVIRONMENTAL  TESTS 
EQUIPMENT  RELIABILITY  TESTS 


ACTUAL  IMPACT  UPON  CANDIDATE  PROGRAM 


MODERATE 

SMALL 

NO 

SMALL 

MODERATE 

SIGNIFICANT 

DECREASE  DECREASE  CHANGE 

INCREASE 

INCREASE 

INCREASE 

a 

□ 

□ 

a 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

a 

a 

□ 

□ 

□ 

□ 

□ 

D 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

a 

□ 

a 

□ 

□ 

□ 

□ 

a 

a 

□ 

a 

□ 

a 

□ 

□ 

□ 

a 

□ 

□ 

a 

□ 

□ 

□ 

□ 

a 

D 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

D 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

a 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

ESTIMATED  IMPACT  WITH  RIW  OPTION  AT  THE 
BEGINNING  OF  ENGINEERING  DEVELOPMENT 


MODERATE 

SMALL 

NO 

SMALL 

MODERATE  SIGNIFICANT 

DECREASE  DECREASE  CHANGE 

INCREASE  INCREASE 

INCREASE 

□ 

a 

□ 

a 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

a 

□ 

□ 

□ 

a 

□ 

□ 

a 

□ 

□ 

o 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

a 

□ 

□ 

o 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

o 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

D 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

D 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

a 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 
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TABLE  1 (Continued) 


QUALITY  ASSURANCE 

QUALITY  ENGINEERING 
INSPECTION 

IN-PROCESS  MONITORING 
TEST  MONITORING 

PROGRAM  MANAGEMENT 
COST  TRACKING 

FINANCIAL  management 
production  cost  ANALYSIS 

ECONOMIC  ESCALLATION  ANALY 
LIFE-CYCLE  COST  ANALYSIS 
PROGRAM  FINANCIAL  CONTROL 

PRODUCT  SUPPORT 

SPARES  REQUIREMENTS 
CONTAINER  DESIGN  A ANALY. 
STE/AGE  DESIGN  A ANALY. 
LOGISTICS  PLANNING 

PRODUCTION  ENGINEERING 

PROOUCIBILITY  ANALYSIS 
MANUFACTURING  engineering 

VALUE  ENGINEERING 

DATA  AND  REPORTS 

CONTRACT  MANAGEMENT 

OTHER  (PLEASE  LIST) 


ACTUAL  IMPACT  UPON  CANDIDATE  PROGRAM 


MODERATE 

SMALL 

NO 

SMALL 

MODERATE  SIGNIFICANT 

DECREASE  DECREASE 

CHANGE 

INCREASE 

INCREASE 

INCREASE 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

a 

a 

□ 

p 

□ 

□ 

□ 

□ 

□ 

O 

D 

D 

a 

□ 

□ 

□ 

□ 

a 

P 

□ 

o 

D 

□ 

a 

□ 

□ 

o 

D 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

a 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

a 

□ 

□ 

a 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

P 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

a 

a 

a 

a 

□ 

□ 

□ 

a 

a 

a 

□ 

a 

□ 

□ 

a 

□ 

□ 

a 

D 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

a 

a 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

ESTIMATED  IMPACT  MITH  R1U  OPTION  AT  THE 
BEGINNING  OF  ENGINEERING  DEVELOPMENT 


MODERATE 

SMALL 

NO 

SMALL 

MODERATE  SIGNIFICANT 

DECREASE 

DECREASE 

Change 

INCREASE 

INCREASE  INCREASE 

a 

a 

□ 

a 

P 

a 

a 

□ 

a 

a 

a 

□ 

D 

□ 

□ 

□ 

a 

a 

a 

a 

□ 

a 

a 

□ 

a 

□ 

a 

a 

a 

□ 

a 

a 

a 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

a 

□ 

□ 

□ 

□ 

a 

a 

□ 

D 

□ 

□ 

a 

a 

n 

p 

□ 

p 

□ 

p 

a 

p 

□ 

p 

a 

p 

p 

p 

p 

p 

n 

p 

p 

p 

a 

p 

p 

p 

p 

p 

□ 

P 

□ 

P 

□ 

□ 

□ 

□ 

□ 

a 

p 

p 

□ 

P 

□ 

p 

□ 

D 

□ 

P 

□ 

p 

p 

P 

□ 

□ 

P 

p 

□ 

□ 

□ 

□ 

□ 

□ 

D 

□ 

p 

□ 

p 

□ 

p 

□ 

a 

p 

p 

p 
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2.  Discussions  with  a small  sample  of  participating 
contractors  indicate  that  there  exist  a number  of  barriers 
which  may  prevent  many  beneficial  development  program  changes 


from  occuring.  In  the  list 

of 

■ 

typical  program  or  product 

design  constraints  presented 

below,  please 

indicate 

those 

which  you  believe  constrain 

changes  which 

would  be  beneficial 

to 

your  future  RIW  option. 

Once  again,  you  are  encouraged  to 

contribute  additional  information. 

No 

Minor 

Maj  or 

Program  Constraint 

Effect 

Constraint 

Constraint 

a. 

Production  Cost  Goal 

rj 

£7 

/~7 

b. 

Development  Schedule 

rj 

£7 

a 

c . 

Development  Funds 

£7 

£7 

£7 

d. 

Performance  Specifications 

C 7 

£7 

a 

e. 

Reliability  Goals 

£7 

£7 

£7  i 

f. 

Military  Specifications: 

Standard  Parts 

£7 

£7 

/_/ 

Reliability  Prediction 

£7 

£7 

/~7 

Environmental  Testing 

£7 

£7 

a 

Configuration  Control 

7£7 

£7 

/~7 

> 

Engineering  Changes 

£7 

£7 

£7  1 

Quality  Assurance 

£7 

£7 

£7 

£7 

£7 

£7 

£7 

£7 

£7 

£7 

£7 

£7 

£7 

£7 

£7 

£7 

£7 

C7 

g. 

Data  Rights 

/~7 

£7 

rj 

h. 

Source  Selection 

£7 

£7 

£7 

i. 

Report  Requirements 

£7 

£7 

£7 

J. 

Other 

£7 

£7 

£7 

£7 

£7 

£7 

£7 

£7 

/~7 

£7 

£7 

7-7  .j 

£7 

/~7 

o 1 
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3.  Prom  the  above  list,  name  the  program  requirement  which 
you  feel  presents  the  greatest  barrier  to  designing  and 
developing  equipment  suitable  for  RIW  application. 


1*.  The  requirements  associated  with  RIW  options  demand  changes 
be  made  in  the  design  of  the  equipment.  Examples  of  such 
changes  are  elapsed-time  indicators,  tamper-proof  box  seals, 
and  warranty  identification  decals.  You  are  requested  to 
list  other  design  changes  which  you  have  made  (or  would 
like  to  have  made)  in  response  to  specific  RIW  option 
requirements . 


Design  Change 


RIW  Requirement  or  Reason 
for  Change 


5.  In  view  of  your  experience  to  date,  name  any  specific 

changes  you  would  propose  in  your  next  development  program 


6.  In  your  opinion,  what  specific  measures  can  either  the 
Government  or  their  contractors  take  during  development 
to  reduce  the  risks  inherent  in  contracting  for  a multi- 
year Reliability  Improvement  Warranty.  


c. 


IMPACT  OF  RIM  AS  A FIRM  CONTRACT  REQUIREMENT 


It  is  the  intent  of  this  part  to  determine  the  potential 
impact  of  RIW  requirements  upon  the  equipment  development 
process  when  it  is  known  at  the  outset  of  engineering  develop- 
ment that  the  equipment  will  be  procured  with  firm  RIW 
obligations.  The  entire  warranty  maintenance  concept  is  to  be 
the  responsibility  of  the  contractor.  The  Government  will, 
however,  have  the  option  to  terminate  or  renew  the  warranty  at 
the  end  of  the  initial  warranty  period  which  should  be  assumed 
to  be  five  years.  The  details  of  the  logistics  support  and 
maintenance  program,  together  with  the  proposed  cost  of  an  RIW, 
will  be  considered  a part  of  the  competitive  production  contract 
source  selection  criteria.  t 

1.  As  in  Part  B above,  you  are  asked  to  indicate  in  the 
following  table  the  impact  of  an  RIW  requirement  upon 
the  development  process.  The  effects  of  two  separate 
operating  conditions  are  again  solicited;  the  first  set 
of  responses  assumes  other  program  and  product  specifica- 
tions continue  to  be  imposed  while  the  second  response 
matrix  is  for  the  situation  where  all  specifications  and 
requirements  are  flexible  and  negotiable  except  for 
critical  performance,  cost,  and  reliability  requirements. 

As  before,  your  additional  comments  or  areas  of  impact 
are  solicited. 


i 
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TABLE  2 


DEVELOPMENT  PROGRAM  ELEMENT 
OESISN 
CIRCUIT 

CARO  AND  BOARO 

MODULE 

SRA 

LRU  OR  FLU 

SYSTEM  ARCHITECTURE 

BUILT-IN  TEST 

TEST  EQUIPMENT 

GROUNO  SUPPORT  EQUIPMENT 

PRODUCTION  TOOLING 

ANALYSIS 

THERMAL 

STRESS 

EMC 

ENVIRONMENT 

FAILURE-MODE-EFFECTS 

RELIABILITY 

MAINTAINABILITY 

PRODUCIBILITY 

OPTIMUM  REPAIR  LEVEL 

COST- PERFORMANCE 

MATERIALS  AND  PURCHASED  PARTS 

PART  SELECTION 

VENDOR  SCREENING 

COST  ESTIMATING  ANO  PRICING 

PROTOTYPE  MANUFACTURE 

COMPONENT  PROCUREMENT 
CARD  OR  BOARD  ASSEMBLY 
MOOULE  ASSEMBLY 
BOX  OR  CHASSIS  ASSEMBLY 
FUNCTIONAL  TEST  AND  CHECKOUT 

EQUIPMENT  TEST 

PART/COMPONENT  ACCEPT.  TEST 
BREADBOARD  TESTS 
BRASS80AR0  TESTS 
EQUIP.  PERFORMANCE  DEMON. 
EQUIPMENT  ENVIRONMENTAL  TESTS 
EQUIPMENT  RELIABILITY  TESTS 


EXPECTED  IWACT  WITH  FIRM  RIW 
REQUIREMENTS  (NO  SPEC  RELIEF) 


MOOCRATE 

SMALL 

NO 

SMALL 

MODERATE  SIGNIFICANT 

DECREASE  DECREASE  CHANGE 

INCREASE 

INCREASE 

INCREASE 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

a 

□ 

p 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

a 

a 

a 

Q 

a 

a 

a 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

a 

□ 

a 

a 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

a 

□ 

a 

□ 

□ 

□ 

□ 

□ 

a 

a 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

D 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

EXPECTED  IMPACT  WITH  FIRM  RIW 
REQUIREMENTS  (FULL  SPEC  RELIEF) 


MODERATE 

SMALL 

NO 

SMALL 

MODERATE  SIGNIFICANT 

DECREASE  DECREASE  CHANGE 

INCREASE 

INCREASE 

INCREASE 

□ 

□ 

□ 

P 

P 

P 

□ 

□ 

□ 

□ 

P 

P 

□ 

□ 

a 

D 

P 

□ 

□ 

□ 

□ 

P 

P 

□ 

□ 

□ 

□ 

P 

P 

P 

□ 

D 

□ 

□ 

P 

P 

□ 

□ 

□ 

P 

P 

P 

□ 

□ 

□ 

P 

P 

P 

□ 

□ 

□ 

P 

P 

P 

□ 

□ 

□ 

P 

P 

P 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

P 

P 

□ 

□ 

□ 

□ 

P 

P 

□ 

□ 

□ 

a 

P 

P 

□ 

□ 

a 

□ 

P 

P 

□ 

□ 

□ 

□ 

P 

P 

□ 

□ 

□ 

p 

P 

P 

□ 

□ 

□ 

p 

P 

P 

□ 

□ 

a 

p 

P 

P 

□ 

□ 

□ 

p 

P 

P 

□ 

□ 

p 

p 

□ 

□ 

□ 

□ 

p 

p 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

p 

p 

□ 

□ 

□ 

□ 

p 

p 

□ 

□ 

□ 

□ 

p 

p 

□ 

P 

□ 

□ 

p 

p 

P 

□ 

□ 

□ 

p 

p 

P 

□ 

□ 

□ 

p 

p 

□ 

D 

P 

□ 

□ 

p 

□ 

□ 

□ 

P 

p 

p 

□ 

□ 

□ 

□ 

p 

p 

P 

P 

□ 

□ 

p 

p 

P 

□ 

□ 

□ 

D 

p 

a 

□ 

TABLE  2 (Continued) 


EXPECTED  IMPACT  WITH  FIRM  RIW 
REQUIREMENTS  (NO  SPEC  RELIEF) 


MODERATE 

SMALL 

NO 

SMALL 

MODERATE 

SIGNIFICANT 

MODERATE 

SMALL 

NO 

SMALL 

MODERATE  SIGNIFICANT  '< 

DECREASE  DECREASE  CHANGE 

INCREASE 

INCREASE 

INCREASE 

DECREASE 

DECREASE 

CHANGE 

INCREASE 

INCREASE 

INCREASE 

QUALITY  assurance 

QUALITY  ENGINEERING 

a 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

a 

INSPECTION 

a 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

a 

IN-PROCESS  MONITORING 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

TEST  MONITORING 

□ 

□ 

a 

a 

□ 

□ 

□ 

□ 

a 

a 

a 

D 

PROGRAM  MANAGEMENT 

COST  TRACKING 

□ 

a 

□ 

□ 

□ 

a 

□ 

a 

D 

□ 

a 

□ 

FINANCIAL  MANAGEMENT 

PRODUCTION  COST  ANALYSIS 

□ 

a 

D 

D 

a 

a 

□ 

□ 

□ 

□ 

□ 

□ 

ECONOMIC  ESCALATION  ANALY. 

□ 

□ 

□ 

□ 

a 

□ 

a 

□ 

□ 

Q 

□ 

□ 

LIFE-CYCLE  COST  ANALYSIS 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

PROGRAM  FINANCIAL  CONTROL 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

a 

a 

□ 

PRODUCT  SUPPORT 

SPARES  REQUIREMENTS 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

D I 

CONTAINER  DESIGN  i ANALY. 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

STE/AGE  DESIGN  S ANALY. 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

LOGISTICS  PLANNING 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

PRODUCTION  ENGINEERING 

PRODUCIBILITY  ANALYSIS 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

MANUFACTURING  ENGINEERING 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

D 1 

VALUE  ENGINEERING 

□ 

□ 

□ 

□ 

□ 

o 

□ 

□ 

□ 

□ 

a 

□ 

DATA  ANO  REPORTS 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

o 

□ 

CONTRACT  MANAGEMENT 

□ 

□ 

□ 

□ 

a 

□ 

□ 

□ 

□ 

□ 

□ 

D 1 

OTHER  (PLEASE  LIST) 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

a 

□ 

a 

g 

□ 

□ 

o 

□ 

□ 

□ 

□ 

□ 

D 

□ 

a 

□ 

□ 

□ 

EXPECTED  IMPACT  WITH  FIRM  RIW 
REQUIREMENTS  (FULL  SPEC  RELIEF) 
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2.  Assuming  a firm  RIW  program  requirement,  list  those  major 
Government''krequirements  and  specifications  which  are 
typically  imposed  upon  a design  and  development  program 
and  which  you  would  either  like  to  have  altered  or  waived. 


3.  Identify  any  specific  changes  in  equipment  design  or 
configuration  which  would  occur  as  a direct  result  of 
knowing  at  the  outset  that  contractor  warranty  provisions 
would  be  part  of  the  production  contract  award. 


(a) 


4.  What  areas  of  new  technology  would  you  like  to  pursue 

(either  on  IR&D  or  company  funds)  to  improve  your  competi- 
tive position  in  developing  equipment  for  warranty 
service? 


What  steps  do  you  believe  the  Government  could  take  during 
the  development  stages  of  a program  to  help  contractors 
optimize  their  designs  for  warranty. 


What  additional  effort,  if  any,  do  you  think  might  be 
pursued  during  development  to  increase  confidence  in 
future  production  equipment  field  reliability  and 
reliability  growth  during  the  initial  phases  of 
deployment.  


What  additional  significant  changes  in  development  program 
plans  or  product  design  can  you  identify  if  contractor 
maintenance  was  planned  throughout  the  lifetime  of  the 
equipment . 


D. 


CASE  STUDY  ANALYSIS 


The  preceeding  sections  have  examined  actual  or  antici- 
pated design  and  development  program  impacts  from  a generalized 
program  point  of  view.  It  is  the  intent  of  this”  section  to 
document  specific  examples  where  the  impact  of  the  RIW  option 
can  be  seen.  One  typical  example  for  both  the  program  and  the 
product  will  be  sufficient  for  our  purposes. 

1.  Describe,  for  your  recent  or  current  candidate  development 
program,  a specific  case  where  the  requirement  for  an 
RIW  option  has  made  an  impact  or  change  in  the  development 
program.  Describe  the  impact  in  terms  of  extra  tasks, 
increased  scope  of  work,  increased  costs,  additional 
schedule  time,  etc.  The  RIW  requirement  which  drove  the 
change  should  also  be  identified. 
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Describe  a change  or  innovation  made  to  the  design  of  the 
candidate  equipment  in  response  to  a specific  RIW  require- 
ment. Identify  the  requirement  and  describe  the  design  or 
configuration  change,  including  some  estimate  of  the 
estimated  production  cost  impact. 


Appendix  E 

LIST  OF  GOVERNMENT  AND 
CONTRACTOR  INTERVIEWS 


GOVERNMENT  AND  CONTRACTOR  INTERVIEWS 


Government  Program  Office 


Interview  Dates 


USN,  NAVELEX  (DTP  EW  Suite) 

USAF,  ESD  (ARN-118  TACAN) 

USAF,  ASD,  (ARC-164  Radar) 

USAF,  ASD,  (B-l  ECM) 

USAF,  ASD  (F-16  SPO) 

USA,  ECOM  (APN-209  Altimeter) 

USA,  ECOM  (ASN-128  Doppler  Navigator) 


18  December  1975 
9 January  1976 
12  January  1976 

12  January  1976 

13  January  1976 
30  January  1976 
1 March  1976 


Contractor 

Hughes  Aircraft  Corp.  (Aerospace  Sys.  Div.) 

Hoffman  Electronics  (NAVCON  Div.) 

Hughes  Aircraft  Corp.  (Ground  Systems  Div.) 

Teledyne/ Ryan  Co. 

General  Dynamics  Corp.  (Electronics  Div.) 

Westinghouse  Electric  Co.  (Defense  & Electronics  Div.) 
Teledyne/ Ryan  Co. 

Kaiser  Aerospace  Corp. 

Collins  Radio  Corp. 

General  Dynamics  Corp.  (Ft.  Worth  Div.) 

Kaiser  Aerospace  Corp. 

Hughes  Aircraft  Corp.  (Aerospace  Groups) 

General  Motors  Corp.  (Delco  Div.) 

Hughes  Aircraft  Corp.  (Ground  Sys.  Div.) 

Hoffman  Electronics  (NAVCON  Div.) 

Hughes  Aircraft  Corp.  (Aerospace  Groups) 

Lear  Siegler  Corp. 

Westinghouse  Electric  Co.  (Defense  Div.) 
Singer-Kearfott  Corp. 

Cutler  Hammer  Corp.  (AIL  Div.) 

Boeing  Aerospace  Corp. 

Raytheon  Co.  (Santa  Barbara  Div.) 

Teledyne/ Ryan  Co. 

General  Dynamics  Corp.  (Electronics  Div.) 

RCA  Inc. 

Rockwell  International  Corp.  (Collins  Radio  Div.) 
Minneapolis  Honeywell  (Defense  Electronics  Div.) 
Magnavox  Co. 

Hoffman  Electric  Co.  (NAVCON  Div.) 

Hughes  Aircraft  Corp.  (Ground  Sys.  Div.) 

Lear  Siegler  Corp. 

Hughes  Aircraft  Corp.  (Aerospace  Div.) 

Raytheon  Co.  (Santa  Barbara  Div.) 


- j 


1 

3 November  1975 

4 November  1975 

5 November  1975 

6 November  1975 

7 November  1975 

14  November  1975 

28  November  1975 
5 December  1975 

22  January  1976 

23  January  1976 
5 February  1976 

9 February  1976 

10  February  1976 

12  February  1976 

13  February  1976 

16  February  1976 

17  February  1976 
9 March  1976 

11  March  1976 

15  March  1976 
23  March  1976 

25  March  1976 

26  March  1976 

29  March  1976 

30  March  1976 

31  March  1976 

27  May  1976 

28  May  1976 

1 June  1976 

2 June  1976 

2 June  1976 

3 June  1976 

4 June  1976 
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ANNOTATED  BIBLIOGRAPHY 


Airborne  Electronic  Equipment  Lifetime  Guarantee , R.  H. 
Myers  and  C.  M.  DeWitt,  III,  Hughes  Aircraft  Company, 
RADC-TR-69-363  (Griffiss  Air  Force  Base,  New  York:  Rome 
Air  Development  Center,  1969). 

Examines  the  feasibility  of  applying  various  types  of 
lifetime  guarantee  plans  to  the  procurement  of  avionics. 
Examines  the  practicality  of  plans  and  guarantee  ramifi- 
cations throughout  system  lifetime.  F-106A  system  data 
were  utilized. 


"An  Analysis  of  Decision  Criteria  for  the  Selection  of  F-16 
Reliability  Improvement  Incentive  Alternatives,"  Thomas  R. 
Kogel  and  Nathan  B.  Mills,  Jr.,  Air  Force  Institute  of 
Technology  thesis  (Ohio:  Wright-Patterson  Air  Force  Base, 

1975). 

Analyzes  F-16  alternatives  from  a government-benefit  point 
of  view.  Methodology  is  developed  to  compute  significant 
costs  to  the  government  over  the  equipment  operational 
lifetime  for  each  alternative.  The  procedures  and  results 
of  the  methodology  are  demonstrated  for  two  selected  F-16 
FLUs. 


Application  of  Design- to-Cost  Acquisition  Policies  to 
Selected  Electronics  Subsystem  Development  Programs , 

C.  David  Weimer,  S-^59  (Arlington,  Va.:  Institute  for 

Defense  Analyses,  1975). 

Analyzes  DoD  experience  in  applying  design-to-cost  acqui- 
sition policies  to  electronic  subsystems.  Fourteen  sub- 
systems representing  three  Services  and  involving  27 
industrial  contractors  were  investigated.  Guidelines  for 
future  policy  development  derived  from  the  findings  were 
recommended  in  the  areas  of  DTC  subsystem  program  planning, 
prediction  of  production  costs  and  equipment  performance, 
reliability  improvement  warranties,  and  subsystem  management. 


**.  Application  of  Reliability  Improvement  Warranty  (RIW)  to 
DoD  Procurements,  Dennis  Jean  Allen  (Monterey,  Calif.: 
Naval  Postgraduate  School,  1975). 

Concludes  that  RIW  may  be  a valuable  tool  for  bring  buyer- 
seller  goals  into  agreement  by  transferring  the  management 
of  costs  to  the  seller. 


5.  Application  of  the  Commercial  Airline  Acquisition  Method- 
ology to  Department  of  the  Navy  Electronic  Equipment 
Acquisition,  L.  J.  Graham,  Pub.  No.  1313-01-1-1447 
(Annapolis,  Md. : ARINC  Research  Corp.,  1975). 

Includes  comparative  analyses  of  the  airline  acquisition 
method  and  the  Navy  acquisition  method,  development  of 
a recommended  acquisition  policy,  preliminary  economic 
analysis,  and  recommendations  for  a pilot  program. 


6.  Audit  Report--Subsy stems  Management , Air  Force  Audit 
Agency,  (1975). 

Discusses  warranty  markings,  storage  of  items  under 
warranty,  warranty  repair,  and  information  systems. 


7.  "Aviation  Supply  Office  FFW/RIW  Case  History  #2,  Abex 

Pump,"  Oscar  Markowitz,  Proceedings  1976  Annual  Reliability 
and  Maintainability  Symposium,  Las  Vegas,  Nev.,  January 
20-22,  1976  (New  York:  Institute  of  Electrical  and  Elec- 

tronics Engineers,  Inc.,  1976). 

Presents  case  history  of  a warranty  application  with  a 
purely  mechanical  equipment. 


8.  Avionics  Reliability  Study,  Lt.  Col.  Ben  H.  Swett,  USAF, 

Phase  I and  II  (Andrews  Air  Force  Base,  Camp  Springs,  Md.: 
Air  Force  Systems  Command,  1973  and  1974). 

Discusses  the  disparity  between  specified  and  operational 
MTBF  of  avionics  equipment. 
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9*  "(The)  Change  in  DoD  Electronics  Acquisition,"  Jacques  S. 
Gansler,  Deputy  Assistant  Secretary  of  Defense  (Materiel 
Acquisition),  Address  Before  the  1972  Winter  General 
Session  of  the  Airlines  Electronic  Engineering  Committee, 
Miami,  Fla.,  December  12,  1974. 

Discusses  three  changes  in  the  approach  to  the  acquisition 
process  for  DoD  weapon  and  electronics  systems — design-to- 
price,  standardization,  and  holding  the  supplier  more 
responsible  for  the  field  reliability  of  his  equipment 
(warranty ) . 


10.  "Considerations  for  Effective  Warranty  Application,"  B.  L. 
Retterer,  Proceedings  1976  Annual  Reliability  and  Maintain 
ability  Symposium,  Las  Vegas,  Nev. , January  20-22,  1976 
(New  York:  Institute  of  Electrical  and  Electronics 

Engineers,  Inc.,  1976). 

Reviews  the  several  forms  that  a warranty  can  take.  Out- 
lines criteria  for  identifying  potentially  effective  appli 
cations.  Presents  a method  for  evaluating  the  economic 
feasibility  of  a warranty. 


11.  "Contractor  Performance/Warranties/Organic  Support:  The 

Failure  Free  Warranty  Experience  at  ASO,"  Thomas  L. 
Schanz,  paper  presented  at  Seminar  on  Interfaces  and 
Relationships  of  Design  to  Cost,  Integrated  Logistics 
Support,  Life  Cycle  Costing,  Washington,  D.C.,  May  14-15, 
1974  (National  Security  Industrial  Association,  1974). 

Discusses  the  elements  of  the  warranty,  the  mutual  sharing 
of  the  risks  and  rewards,  the  estimate  of  reliability,  and 
the  real-life  influences  on  that  reliability. 


12.  "(A)  Contractor  View  of  Warranty  Contracting,"  William  J. 

Bonner,  Proceedings  1976  Annual  Reliability  and  Maintain- 
ability Symposium,  Las  Vegas,  Nev.,  January  20-22,  1976 
(New  York:  Institute  of  Electrical  and  Electronics 

Engineers,  Inc.,  1976). 

Presents  a set  of  typical  warranty  contract  provisions 
and  then  proceeds  to  discuss  the  challenges  offered  to 
the  contractor  by  those  provisions. 
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13.  (A)  Cost  Effectiveness  Study  of  Air  Force  v a.  Contract 

Maintained  Non-Tactical  Radio  Systems,  Gerald  E. 
Bronnenberg  (Maxwell  Air  Force  Base,  Ala. : Air 

University,  1972). 

Compares  the  costs  of  contract  nontactical  radio  mainte- 
nance with  Air  Force  maintenance. 

14.  "Cost  of  Reliability  Improvement,"  Avery  H.  Heuesh,  Pro- 

ceedings 1969  Annual  Symposium  on  Reliability , Chicago, 
111.,  January  21-23,  1969  (New  York:  Institute  of 

Electrical  and  Electronics  Engineers,  Inc.,  1969). 

Examines  systematically  the  contributions  of  individual, 
independent  methods  for  reliability  improvement  and 
computes  their  "efficiencies"  using  data  from  a number  of 
major  reliability  programs. 

15.  "Designing  for  LCC,"  William  H.  Boder,  Defense  Management 
Journal,  January  1976. 

Discusses  the  Magnavox  experience  with  LCC  on  their  ARC- 
164  radio. 


16.  (The)  Development  and  Analysis  of  RIW  and  COD  Provisions 

for  the  Air  Combat  Fighter  (ACF)  Aircraft,  G.  Harrison,  Pub. 
No.  1264-01-1-1370  (Annapolis,  Md. : ARTNC  Research  Corp., 

1975). 

Describes  the  chronology  of  events  and  the  decision 
rationale  in  the  evolution  of  life-cycle-cost  controls 
for  the  air  combat  fighter.  Includes  "lessons  learned" 
summary. 


17.  Electronice-X:  A Study  of  Military  Electronics  With 

Particular  Reference  to  Cost  and  Reliability , in  2 
vols.,2:  "Complete  Report,"  Howard  P.  Gates,  Jr.,  et  al., 

R-195  (Arlington,  Va. : Institute  for  Defense  Analyses, 

1974). 

Identifies  the  current  DoD  and  industrial  policies, 
procedures,  and  practices  in  development,  production,  and 
operational  support  that  most  significantly  influence  the 
cost  and  reliability  of  military  electronics.  Recommends 
changes  to  reduce  and  control  cost  and  to  improve 
reliability.  The  report  concentrates  on  five  major. 
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high- impact  areas:  (1)  data  collection  and  feedback,  (2) 
requirements,  (3)  competition  and  management  options,  (4) 
reliability  enhancement,  and  (5)  maintenance  training. 
Numerous  other  areas  are  discussed,  and  detailed  recommenda- 
tions are  made  in  regard  to  each. 


18.  Evaluation  of  F-16  Subsystem  Options  Through  the  Use  of 
Mission  Completion  Success  Probability  and  Designing  to 
System  Performance/Cost  Models,  A.  M.  Doman  and  A.  G. 
Dunkerly,  Air  Force  Institute  of  Technology  thesis  (Ohio: 
Wright-Patterson  Air  Force  Base,  1975). 

Analyzes  F-16  subsystems  and  determines  optional  strategies 
for  RIW,  RIW/MTBF,  and  TLSC  options  in  terms  of  mission 
completion  success  probabilities  and  costs. 


19.  "Factors  on  Balancing  Government  and  Contractors  Risk  with 
Warranties,"  Russell  R.  Shorey,  Proceedings  1976  Annual 
Reliability  and  Maintainability  Symposium , Las  Vegas,  Nev., 
January  20-22,  1976  (New  York:  Institute  of  Electrical  and 

Electronics  Engineers,  Inc.,  1976). 

Discusses  the  DoD  background,  new  incentive  techniques, 
warranty  pricings,  warranty  application,  and  LCC  implications. 


20.  "Failure  Free  Warranty — 5 Year  Results,"  R.P.  Wilcox, 

NAECON  '74  RECORD  (New  York:  Institute  of  Electrical  and 

Electronic  Engineers,  Inc.,  197*0. 

Discusses  the  Failure  Free  Warranty  (FFW)  concept  and  con- 
cludes that  the  original  program  was  a total  success. 


21.  Government  Contract  Warranties , Government  Contracts  Mono- 
graph No.  2 (Washington,  D.C.:  George  Washington  Uni- 

versity, 1961). 


Includes  four  papers:  (1)  "Warranties  under  the  General 

Law  of  Sales — Some  Relationships  to  Government  Contract 
Law,"  by  John  W.  Whelan;  (2)  "Government  Contract  Warranties," 
by  Frederick  Sess;  (3)  "An  Industry  Attitude  Toward  Govern- 
ment-Required Warranty  Clauses,"  by  John  F.  Carr;  and  (4) 

"Army  Contractual  Warranties — The  Quest  for  Quality,"  by 
Donald  E.  Miller. 


22. 


"Government  Depot  Maintenance  Warranties,"  Russell  M.  Genet, 
Proceedings  1976  Annual  Reliability  and  Maintainability 
Symposium , Las  Vegas,  Nev.,  January  20-22,  1976  (Institute 
of  Electrical  and  Electronics  Engineers,  Inc.,  1976). 

Discusses  the  concept  of  government  depots'  offering 
maintenance  warranties. 


23.  Government  Depot  Maintenance  Warranties , Don  E.  Hunt, 
Russell  M.  Genet,  and  Theodore  Crosier,  AGMC  74-019 
(Ohio:  Aerospace  Guidance  & Metrology  Center,  Newark 

Air  Force  Station,  1974). 

■ I 

Introduces  the  concept  of  government  depot  maintenance 
warranties  and  discusses  their  advantages  and  disadvantages, 
what  a warranty  might  consist  of,  and  how  it  might  work. 


24.  Guidelines  for  Application  of  Warranties  to  Air  Force 

Electronic  Systems , Harold  S.  Balaban  and  Bernard  L.  Retterer, 
1500-01-1-1451  (Annapolis,  Md. : ARINC  Research  Corp.,  1975). 

Reviews  basic  types  of  warranty  plans.  Provides  informa- 
tion on  the  applicability  of  various  types,  developing 
a set  of  terms  and  conditions,  administrative  procedures 
for  developing  and  implementing  a warranty,  and  evaluation 
procedures  for  monitoring  warranty  performance.  A life- 
cycle  cost  model  is  presented.  Includes  a case  study  of 
the  AN/ARN-118  TACAN  set.  Sample  warranty  provisions,  a 
user's  manual,  and  other  reference  material  are  also 
provided. 


25.  "Industry  Application  of  Failure  Free  Warranty  Techniques," 
Ralph  P.  Wilcox,  presented  at  8th  Annual  Joint  Services 
Data  Exchange  Conference  for  Inertial  Systems,  held  at 
Charles  Stark  Draper  Laboratory,  Cambridge,  Mass.,  August 
19-21,  1974. 

Discusses  both  the  general  application  of  FFWs  and  5-year 
results  for  the  Lear  Siegler  gyro  as  a case  study. 


26.  Industry  Sub  Group  Report  on  RAM  Incentives  and  Warranties , 
R.  H.  Wain,  et  al.  (Washington,  D.C.:  American  Defense 

Preparedness  Association,  1975)* 

Reports  on  background  information  obtained  from  U.S.  Army 
Electronics  Command  on  methods  and  techniques  for  expanding 
use  of  RAM  incentives  and  warranties. 
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27.  Interim  Guidelines , Reliability  Improvement  Warranty  (RIW)t 
U.S.  Air  Force,  Hq.,  Directorate  of  Procurement  Policy 
(AF/LGP),  DCS/Systems  and  Logistics  (197*0. 

Provides  guidance  with  respect  to  RIW  application  criteria; 
funding  of  RIWs ; essential  elements  to  be  included  in  an 
RIW  contract  clause;  determination  of  cost  effectiveness  of 
use  of  an  RIW  provision;  and  evaluation  approaches  that  can 
be  used  to  assess  the  cost  effectiveness  of  an  RIW  after  it 
has  been  implemented. 


28.  Long  Term  Service  Warranty  Contracts-- A Case  Example  of 
Gyroscopes  Purchased  Under  Warranty , Joseph  L.  Higgins, 

Air  Force  Institute  of  Technology  (Ohio:  Wright-Patterson 

Air  Force  Base,  1972). 

Analyzes  Air  Force  organizational  and  managemental  aspects 
of  long-term  service  warranty  contract  provisions  using  an 
Air  Force  contract  for  an  aircraft  gyroscope  as  a vehicle 
for  analysis.  Two  areas  of  analysis  were  warranty  funding 
and  warranty  data  requirements. 


29.  Magnuson-Moss-Warranty  Federal  Trade  Commission  Improvement 
Act , Public  Law  93-637;  88  STAT.  2183. 

Act  provides  minimum  disclosure  standards  for  written 
consumer  product  warranties,  defines  minimum  Federal  con- 
tent standards  for  such  warranties,  and  amends  the  Federal 
Trade  Commission  Act  in  order  to  improve  its  consumer 
protection  activities. 


30.  "Managing  Downstream  Weapons  Acquisition  Costs — Some  Key 
Elements,"  Russell  R.  Shorey,  Defense  Management  Journal , 
January  1976. 

Discusses  O&S  cost  visibility,  O&S  cost-related  thresholds, 
design  trades  to  minimize  LCC,  contract  and  other  incentives 
to  reduce  O&S  costs,  and  logistics  alternatives. 


31.  Methods  of  Acquiring  and  Maintaining  Aircraft  Engines 

(Washington,  D.C.:  Logistics  Management  Institute,  1972). 

Discusses  a mathematical  model  constructed  to  compare 
military  and  commercial  differences  in  practice  that  would 
affect  the  use  of  warranties.  The  report  recommends 
against  the  use  by  DoD  of  aircraft  engine  warranties  but 
encourages  continued  research  and  analysis  into  specific 
applications. 
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(A)  Model  for  Contract-  Pricing  for  Use  by  Government  Depots 
in  Conjunction  with  the  Use  of  Government  Depot  Warranties 
in  Multi-Year  Contracting  at  Fixed  Prices , Don  E.  Hunt, 

AGMC  7^-01311  (Ohio:  Aerospace  Guidance  and  Metrology 

Center,  Newark  Air  Force  Station,  197*0. 

Examines  the  fundamental  economic  considerations  of  the 
maintenance  warranty  concept  embodied  in  fixed-pricing 
and  multi-year  contracting.  A model  is  developed  that 
incorporates  these  concepts  into  a government  depot 
maintenance  warranty. 


"(A)  Monte  Carlo  Risk  Analysis  of  Life  Cycle  Cost  Predic- 
tion," Samuel  B.  Graves,  Air  Force  Institute  of  Technology 
thesis  (Ohio:  Wright-Patterson  Air  Force  Base,  1975). 

Investigates  the  uncertainties  involved  in  the  prediction 
and  measurement  of  life-cycle  costs.  The  logistics  sup- 
portability  incentives  in  the  F-16  contract  are  analyzed 
in  light  of  the  stochastic  uncertainties  of  prediction 
and  measurement  of  logistics  support  costs.  A Monte  Carlo 
simulation  model  is  applied  to  determining  appropriate 
contractor  rewards  or  penalties. 


"(A)  New  Approach  to  Long  Range  Fixed  Price  Warranty  Within 
Operational  Environments — Buyer/User,"  Oscar  Markowitz, 
Annals  of  Reliability  and  Maintainability  (1971)  (New 
York:  Institute  of  Electrical  and  Electronics  Engineers, 

Inc.,  1971),  PP.  252-58. 

Discusses  the  Lear  Siegler/Navy  gyro  contract  from  the 
buyer-user  viewpoint. 


"(An)  Overview  of  RIW  Procurement,"  Harold  Balaban,  paper 
presented  at  Joint  Logistics  Commanders  Conference,  Airlie 
House,  Warrenton,  Va. , May  1975. 

Presents  guidance  for  structuring  the  warranty.  Discusses 
the  warranty  concept,  RIW  terms  and  conditions,  and  RIW 
development . 


36. 


"(A)  Practical  Life  Cycle  Cost/Cost  Ownership  Type  Procure- 
ment Via  Long  Term/Multi-Year  'Failure  Free  Warranty'  (FFW) 
Showing  Trial  Procurement  Results,"  James  C.  Harty,  Annals 
of  Reliability  and  Maintainability  (1971)  (New  York:  Instl- 

tue  of  Electrical  and  Electronics  Engineers,  Inc.,  1971), 
pp.  241-51. 

Discusses  the  Lear  Siegler/Navy  gyro  contract  from  the 
designer-supplier  viewpoint. 


37.  Proceedings  of  Failure  Free  Warranty  Seminar  (Philadelphia: 
U.S.  Navy  Aviation  Supply  Office,  1973). 

Reports  on  such  FFW  subject  areas  as:  introduction  to  FFW, 

contractual  considerations,  case  studies,  overhaul  shop 
practices.  Navy  3M  data,  and  analytical  techniques. 


38.  Project  ACE-- Findings  and  Action  Plans , Air  Force  Systems 
Command  (Andrews  Air  Force  Base,  Camp  Springs,  Md.,  1973). 

Project  ACE  (Acquisition  Cost  Evaluation)  studied  opportuni- 
ties for  achieving  drastic  reductions  in  weapon  system 
acquisition  and  ownership  costs.  Findings  relating  to 
reliability  improvement  and  long-term  warranties  are 
included. 


39.  Reliability  Acquisition  Cost  Study , Salvatore  P.  Mercurio 
and  Clyde  W.  Skaggs,  General  Electric  Company,  RADC-TR-73- 
334  (Griff iss  Air  Force  Base,  New  York:  Rome  Air  Develop- 

ment Center,  1973). 

Develops  basic  relationships  capable  of  determining  and 
predicting  the  acquisition  costs  attributable  to  equip- 
ment reliability.  Relationships  were  developed  using  data 
from  two  manufacturers  on  ten  equipments  covering  three 
reliability  elements. 


40.  "Reliability  Augmentation  Maintenance  Program,  Altimeter 
Receivers-Transmitters  (RAMPART) -Warranty  Program,"  James 
J.  Larkin,  Proceedings  of  Quarterly  Meeting  of  the  Life 
Cycle  Cost  Task  Group  of  the  Joint  Services  Data  Exchange 
for  Inertial  Systems , San  Diego,  Calif.,  February  24-26,  1976. 

Uses  vugraphs  to  discuss  the  depot  overhaul  of  the  APN-141 
altimeter  receiver-transmitter.  Vugraphs  cover  the  RIW 
concept,  benefits,  advantages,  disadvantages,  cost-effective 
design  changes,  rework  procedures,  changes  in  maintenance 
concepts,  and  LCC  savings. 
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41.  "Reliability  Improvement  Warranty:  An  Experimental  Logis- 

tics Support  Concept,"  Ronald  A.  Mlinarchik,  Proceedings 
of  the  14th  Annual  US  Army  Operations  Research  Symposium , 

Fort  Lee,  Va.,  November  1975* 

Discusses  what  RIW  is,  how  concept  was  developed,  policy  ' 

that  governs  it,  potential  benefits,  how  it  works,  and 
some  examples  of  its  use  by  the  Services.  Highlights  of 
the  CONUS  NAV  Program  RIW  clause  are  discussed.  An  over- 
view of  RIW  evaluation  techniques  is  presented  with 
emphasis  on  cost  effectiveness,  reliability  growth,  risk, 

I and  suitability.  RIW  problems  and  solutions  are 

presented. 
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42. 


43. 


44. 


45. 


Reliability  of  Military  Electronic  Equipment , Advisory 
Group  on  Reliability  of  Electronic  Equipment  (AGREE),  Office 
of  the  Assistant  Secretary  of  Defense  (Research  and  Engi- 
neering) (1957). 

The  landmark  report  on  reliability  of  electronic  equipment. 


"Reliability  Planning  and  Management  (RPM),"  J.  D.  Selby 
and  S.  G.  Miller,  presented  at  American  Society  for  Quality 
Control  Seminar,  Niagara  Falls,  N.Y.,  September  1970. 

Explains  use  of  RPM,  a management  tool  for  bridging  the  gap 
between  stated  reliability  requirements  and  implementation 
planning. 


"Reliability  Tasks  vs.  Product  Reliability,"  David  I.  Troxel, 
Proceedings  1969  Annual  Symposium  on  Reliability , Chicago, 
111.,  January  21-23,  1969  (New  York:  Institute  of  Electri- 

cal and  Electronics  Engineers,  Inc.,  1969). 

Analyzes  various  reliability  tasks  with  regard  to  their 
interrelationship  and  applicability  to  meeting  requirements. 


"Reliability:  What  Happens  If...?,"  Ervin  F.  Taylor, 

Proceedings  1969  Annual  Symposium  on  Reliability , Chicago, 
111.,  January  21-23,  1969  (New  York:  Institute  of  Elec- 

trical and  Electronics  Engineers,  Inc.,  1969). 

Examines  both  the  quantitative  and  qualitative  approaches 
to  a reliability  program.  Quantitative  disciplines  include 
prediction,  risk  analysis,  and  measurement.  Qualitative 
disciplines  include  design  review,  FMECA,  parts-material 
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review,  specifications  and  standards,  failure  evaluation, 
configuration  control,  maintenance  programs,  and  integrated 
test  programs. 


"Source  Selection  and  Contracting  Approach  to  LCC  Manage- 
ment," J.  W.  Stansberry,  Defense  Management  Journal , 
January  1976. 

Discusses  RIWs  briefly  along  with  LCC  and  source 
selection. 


"Study  of  Warranted  Items,"  Robert  E.  Black,  Jr.,  Manage- 
ment & Intern  Project,  Defense  Supply  Agency,  Defense 
Contract  Administration  Services,  September  1972. 

Reports  on  a general  study  of  warranties  in  government 
contracts.  Covers  procurement  regulations  and  warranty 
knowledge,  identification  processing,  and  administration. 


Techniques  for  Selecting  and  Analyzing  Reliability  Improve- 
ment Warranties , Joseph  A.  Bizup  and  Randall  R.  Moore, 
R-7505  (Washington,  D.C.:  Naval  Weapons  Engineering  & 

Support  Activity,  1975). 

Presents  selection  criteria  for  RIW  candidates.  Techniques 
for  risk  and  economic  analysis  are  developed.  Three  con- 
tracts are  analyzed  in  detail  with  respect  to  RIW  impact 
on  item  procurement  and  management.  The  cost-effectiveness 
of  two  contracts  is  discussed  to  indicate  areas  of  probable 
life-cycle-cost  savings. 


(The)  Use  of  Warranties  for  Defense  Avionics  Procurement , 

H.  Balaban  and  B.  Retterer,  ARINC  Research  Corporation, 
RADC-TR-248  (Griffiss  Air  Force  Base,  New  York:  Rome  Air 

Development  Center,  1973). 

Investigates  potential  benefit  of  using  warranty  agreements 
as  part  of  military  avionics  procurements.  Interviews  were 
conducted  with  warranty  users,  and  a life-cycle  cost  model 
was  formulated.  The  major  conclusion  is  that  a properly 
constituted  and  applied  warranty  can  yield  significant 
reliability  and  life-cycle-cost  benefits  and  that  broader 
use  of  warranties  is  advisable. 
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50.  "Warranties  as  a Life  Cycle  Cost  Management  Tool,"  C.  R. 
Knight,  Defense  Management  Journal , January  1976. 

Discusses  LCC  management,  basic  limitations  in  achieving 
reliability  of  the  government  customer,  the  burden  on  the 
manufacturer,  the  responsibility  for  total  cost,  the 
limitations  of  warranties,  and  the  associated  risk  of 
warranties. 


51.  "Warranties  as  a Life-Cycle-Cost  Management  Tool,"  C.  R. 
Knight,  EASCON  '74  Record  (New  York:  Institute  of  Elec- 

trical and  Electronics  Engineers,  Inc.,  1974). 

Discusses  use  of  warranties  to  control  life-cycle  costs  of 
military  equipments  as  an  alternative  to  standard  procure- 
ment approaches. 


52.  "Warranties:  DoD  Initiates  Trial  of  Reliability  Improve- 

ment Warranties  in  Buying  Electronic  Equipment,"  Federal 
Contracts  Report , November  10,  1974  (Washington,  D.C. : 

The  Bureau  of  National  Affairs). 

Discusses  OSD  memorandum  that  provides  guidelines  for  new 
RIW  to  be  applied  in  electronics  field. 


53*  Warranties  for  Military  Avionics  Procurement , R.  E.  Adel, 
(Hawthorne,  Calif.:  Northrop  Electronics,  1975). 

Presents  history  of  warranties  and  discussion  of  commercial 
airline  experience.  Discusses  limited  applications  to 
military  programs  and  provisioning  details  of  RIWs. 


54.  "Warranties  Under  Government  Contracts,"  J.  G.  Twomey, 
Insurance  Law  Journal , August  1970,  pp.  464-71. 

Discusses  the  need  for  articulation  of  a warranty  policy 
on  government  contracts  that  would  provide  the  government 
the  protection  it  needs  and  can  afford,  and  that  would 
provide  uniform  fair  treatment  for  its  contractors,  sub- 
contractors, and  suppliers. 


P-12 


"Warranty  Cost  Estimates  for  Avionic  Subsystems,"  P.  0. 
Nerber,  Proceedings  1969  Annual  Sympoaium  on  Reliability , 
Chicago,  111.,  January  21-23,  1969  (New  York:  Institute 

of  Electrical  and  Electronics  Engineers,  Inc.,  1969)* 

Presents  a general  and  analytical  technique  for  estimating 
the  failure  and  cost  exposures  associated  with  a warranty 
commitment  on  avionic  systems. 


"Warranty  Guarantee  and  Contractor  Exposure  to  Cost 
Penalties,"  Robert  McGinns,  Proceedings  of  Quarterly  Meet- 
ing of  the  Life  Cycle  Cost  Task  Group  of  the  Joint  Services 
Data  Exchange  for  Inertial  Systems , San  Diego,  Calif., 
February  24-26,  1976. 

Discusses  the  types  of  cost  penalties,  adjustments,  and 
other  considerations  that  were  identified  after  review  and 
analysis  of  a specific  RIW  contract.  Includes  for  each 
specific  type  of  consideration,  the  amount  of  contractor 
risk  or  exposure  to  cost  penalties,  a parametric  analysis, 
and  identification  of  critical  areas  for  management 
attention. 


"Warranty  Procurement — A Case  History,"  Harold  S.  Balaban 
and  Frederick  J.  Nohmer,  Proceedings  of  the  197 S Annual 
Reliability  and  Maintainability  Symposiumt  Washington,  D.C., 
January  1975  (New  York:  Institute  of  Electrical  and 

Electronics  Engineers,  Inc.,  1975). 

Gives  a case  history  of  the  development  of  a complete 
warranty  provision  for  the  Air  Force  ARN-118  TACAN. 


58.  World  Airline  Suppliers  Guide  (Washington,  D.C.:  Air 

Transport  Association  of  America).  • 

Covers  commercial  airline  warranties  (chapter  10). 


